Telemedicine system with dynamic imaging

ABSTRACT

A telemedicine system with dynamic imaging is disclosed herein. In some embodiments, the telemedicine system comprises a laser imaging and treatment apparatus, and associated systems and methods that allow a physician (e.g., a surgeon) to perform laser surgical procedures on an eye structure or a body surface with a laser imaging and treatment apparatus disposed at a first (i.e. local) location from a control system disposed at a second (i.e. remote) location, e.g., a physician&#39;s office. Also, in some embodiments, communication between the laser imaging and treatment apparatus and control system is achieved via the Internet®. Further, in some embodiments, the telemedicine system includes a dynamic imaging system and/or a facial recognition system that verifies the identity of a patient, and is capable of being used for other important applications, such as tracking and analyzing trends in a disease process.

CROSS-REFERENCE TO RELATED APPLICATIONS

This patent application is a continuation-in-part of U.S. patentapplication Ser. No. 15/897,085, filed Feb. 14, 2018; which claims thebenefit of U.S. Provisional Application No. 62/459,009, filed on Feb.14, 2017, U.S. Provisional Application No. 62/549,941, filed on Aug. 24,2017, and to U.S. Provisional Application No. 62/563,582, filed on Sep.26, 2017; and U.S. patent application Ser. No. 15/897,085 is acontinuation-in-part of U.S. patent application Ser. No. 15/368,759,filed Dec. 5, 2016, now U.S. Pat. No. 9,931,171; which is acontinuation-in-part of U.S. patent application Ser. No. 14/715,325,filed May 18, 2015, now U.S. Pat. No. 9,510,974; which is acontinuation-in-part of U.S. patent application Ser. No. 13/865,901,filed Apr. 18, 2013, now U.S. Pat. No. 9,037,217; which is acontinuation-in-part of U.S. patent application Ser. No. 13/573,100,filed Aug. 20, 2012, now U.S. Pat. No. 8,903,468; which is acontinuation-in-part of U.S. patent application Ser. No. 12/925,518,filed Oct. 22, 2010, now U.S. Pat. No. 8,452,372; which claims thebenefit of U.S. Provisional Application No. 61/455,111, filed Oct. 13,2010; the disclosure of each of which is hereby incorporated byreference as if set forth in their entirety herein.

This patent application is also a continuation-in-part of PatentCooperation Treaty Patent Application No. PCT/US2018/041958, filed Jul.13, 2018; which claims the benefit of U.S. Provisional Application No.62/532,098, filed on Jul. 13, 2017, U.S. Provisional Application No.62/671,525, filed on May 15, 2018, U.S. Provisional Application No.62/549,941, filed on Aug. 24, 2017, and to U.S. Provisional ApplicationNo. 62/563,582, filed on Sep. 26, 2017; the disclosure of each of whichis hereby incorporated by reference as if set forth in their entiretyherein.

FIELD OF THE INVENTION

The present invention relates to methods and systems for telemedicineand for laser treatment of the eye or a body surface. More particularly,the present invention relates to a telemedicine system with dynamicimaging and/or facial recognition capabilities.

BACKGROUND OF THE INVENTION

As is well known in the art, various eye disorders, such as diabeticretinopathy, vascular occlusion, neovascularization and age maculardegeneration, can, and in most instances will, have an adverse effect onthe retina. Indeed, if not treated at the appropriate stage, noteddiseases, particularly, diabetic retinopathy, can lead to severe lossesin vision.

Various methods and systems have thus been developed to aid in thediagnosis of the noted eye diseases. The method often employed by an eyecare specialist, such as an ophthalmologist, is to examine the ocularfundus (the inside back surface of the eye containing the retina, bloodvessels, nerve fibers, and other structures) with an ophthalmoscope.

The ophthalmoscope is a small, hand-held device, which, whenappropriately positioned, shines light through a subject's pupil toilluminate the fundus. By properly focusing the light reflected from thesubject's fundus, an examiner can observe the fundus structures.

As is well known in the art, examination of the ocular fundus can alsobe achieved using a fundus or slit lamp camera. Illustrative are theapparatus and systems disclosed in U.S. Pat. Nos. 5,713,047, 5,943,116,5,572,266, 4,838,680, 6,546,198, 6,636,696, 4,247,176, 5,742,374 and6,296,358.

Various method and systems have also been developed to treat eyedisorders, such as diabetic retinopathy, glaucoma and age maculardegeneration. One known method of treating the noted eye disorders, aswell as retinal detachment, is laser coagulation of predeterminedbiological structures of the eye, such as the retina.

As is well known in the art, during laser coagulation of an eyestructure, laser energy is transmitted to the structure to effectcoagulation thereof. A typical laser coagulation system thus includes alaser energy or beam source, such as a beam projector, a slit imageprojector or lamp for forming a slit image on the eye, and observationequipment for observing the slit image and laser spot(s) in the eye.Illustrative are the laser coagulation systems disclosed in U.S. Pat.Nos. 4,759,360 and 4,736,744.

A major drawback associated with each of the noted conventional systems,as well as most known laser coagulation systems (and associatedmethods), is that the conventional slit lamp systems require a contactlens to neutralize the refractive power of the cornea. A contact lens isalso necessary to provide a variable field of view of the retina up to130°.

As is well known in the art, the contact lens must be appropriatelypositioned on the surface of the cornea and held at the desired positionby the specialist, e.g., surgeon, while looking through the slit lampmicroscope.

During this conventional laser coagulation procedure, the contact lensis positioned on the cornea, and held in position by the surgeon so asto permit the surgeon to view the retina through the slit lampmicroscope during the laser application to the retina. In allconventional contact systems, the field of view is limited (e.g.,maximum 50-60 degrees) and the surgeon is required to move the contactlens from one side of the eye to the other side of the eye during theprocedure, and the patient is also required to move his or her eye, inorder to permit the surgeon to see the peripheral retina.

There are several drawbacks associated with the use of a contact lensduring laser coagulation. A major drawback is that the use of a contactlens requires topical anesthesia and a dilated pupil for laserapplication. As is well known in the art, a contact lens can, and inmany instances will, cause corneal abrasion on an anesthetized cornea.

A further drawback associated with conventional laser coagulationprocedures is that the laser procedures are dependent on the steadinessof the physician's hands and the subject's head.

Another apparatus that is often used for laser energy delivery to theperipheral retina is the indirect ophthalmoscope. Use of the indirectophthalmoscope requires a physician to hold an appropriate convex lensin front of the eye (pupil) with one hand to focus the laser beam on theretina, while the eye is indented with another hand to bring theperipheral retina into the field of view for laser application.

In the indirect ophthalmoscopy technique, which is an alternativeconventional method, the physician (i.e., surgeon) does not place acontact lens on the cornea, but rather he or she has to indent theperipheral part of the eye with an indenter (or scleral depressor) tobring the peripheral retinal areas into view, and additionally, thepatient has to move the eye side to side.

Although laser delivery with an indirect ophthalmoscope eliminates theneed for a contact lens, there are still drawbacks and disadvantagesassociated with use of an indirect ophthalmoscope. A major drawback isthat during laser delivery (and, hence, coagulation of a desired eyestructure), the ophthalmoscope is often carried on the physician's headfor 30-60 mins. This extended period causes extreme fatigue for thephysician.

The indentation of the eye for the extended period is also veryunpleasant for the subject or patient.

A further drawback associated with the use of an indirect ophthalmoscopefor laser coagulation is that the indirect ophthalmoscope does notprovide a retained record or documentation for future evaluation.Further, in most instances, the subject typically requires subsequentfundus photography.

None of the abovedescribed conventional methods are suitable for remotelaser application because they are limited in their field of view(typically 50-60 degrees). Also, the eye movement that is needed withthese systems to view the entire retina renders them unsuitable forremote applications.

It would thus be desirable to provide non-contact systems and methodsfor laser coagulation of eye structures to treat eye disorders, and arecapable of being effectively utilized to treat patients located at aremote site.

It is therefore an object of the present invention to providenon-contact systems and methods for laser coagulation of eye structuresthat substantially reduce or overcome the noted drawbacks anddisadvantages associated with conventional contact-based lasercoagulation systems and methods.

It is another object of the present invention to provide non-contactapparatus, systems and methods for laser imaging and coagulation of aneye structure.

It is yet another object of the present invention to provide non-contactapparatus, systems and methods for laser imaging and coagulation of theretina and its periphery to treat retina and choroideal disorders and/ordiseases.

It is still another object of the present invention to provide a systemfor remote laser treatment of an eye structure or a body surface that isfurther capable of performing photodynamic therapy on a patient.

It is yet another object of the present invention to provide atelemedicine system or a remote laser treatment system with dynamicimaging that more accurately verifies the identity of a patient, and iscapable of being used for other important applications, such as trackingand analyzing trends in a disease process.

It is still another object of the present invention to provide a remotelaser treatment system that determines the geographical location of thelocal laser generation unit of the system so as to ensure that the locallaser generation unit has not been improperly intercepted or stolen byunauthorized individuals.

SUMMARY OF THE INVENTION

The present invention is directed to laser imaging and coagulationapparatus, systems and methods that allow an eye specialist, e.g., anophthalmologist or surgeon, to perform laser surgery on an eyestructure, e.g. retina, with an integral laser imaging and coagulationapparatus disposed at a first (i.e. local) location from a controlsystem disposed at a second (i.e. remote) location, e.g., a physician'soffice. The laser imaging and coagulation apparatus, systems and methodsof the invention thus make it possible for an ophthalmologist to screenand perform laser surgery to treat various eye disorders, including,without limitation, diabetic retinopathy, vascular occlusion,neovascularization and age macular degeneration from a geographicallyremote location. The laser imaging and treatment system described hereinmay also be used to perform photodynamic therapy on a patient.

In one embodiment of the invention, the laser coagulation systemincludes

(i) at least a first laser-imaging system disposed at a first location,the first laser-imaging system including a first laser-imagingapparatus, a photoacoustic system, a first processor and a local controlmodule,

the first laser-imaging apparatus including a wide angle digital imageacquisition system for acquiring digital images of a subject's eyestructure and a laser generation system for transmitting an aiming laserbeam and at least a first coagulation laser beam to the eye structure,the first coagulation laser beam having a first laser energy,

the photoacoustic system being configured to measure temperature of eyestructure tissue subjected to the first laser energy,

the local control module including local operation, local operation andperformance simulation and local safety and verification sub-modules;and

(ii) a central control system disposed at a remote site, the centralcontrol system including a second processor and a remote control module,

the remote control module including remote operation, remote operationand performance simulation, and remote safety and verificationsub-modules,

the remote operations sub-module being configured to facilitatecommunications between a remote physician and the remote processor, andperform a laser coagulation procedure on the eye structure in an actualcontrol mode.

In some embodiments of the invention, the local operation sub-module isconfigured to acquire at least a first eye structure image from thedigital image acquisition system and transmit the first eye structureimage to the remote site, receive a target laser transmission area andlaser transmission parameters from a remote physician, apply an activecontour algorithm to partition the first eye structure image into a gridmap, perform a scatter laser (focal or grid) coagulation of the eyestructure under the remote physician's command, acquire a plurality ofpost-procedure eye structure images, and transmit the post-procedure eyestructure images to the remote site for evaluation and verification oftreatment.

In some embodiments, the remote operations sub-module is furtherconfigured to execute a virtual treatment of the eye structure andperform a test surgical procedure in association with the localoperation and performance simulation sub-module.

In some embodiments, the remote operation and performance simulationsub-module is configured to test performance parameters of the localoperation module and perform virtual treatment of the eye structure bythe remote physician.

In some embodiments of the invention, the photoacoustic system isconfigured to control the laser generation system.

In one embodiment of the invention, the eye structure comprises theretina.

In some embodiments of the invention, the laser coagulation system alsoincludes eye tracking means for tracking movement of the eye.

In some embodiments of the invention, the laser coagulation system alsoincludes facial recognition means for identifying and/or verifying theidentity of the subject.

In one embodiment, communication by and between the central controlsystem and the laser-imaging apparatus is achieved via the Internet®.

In another embodiment, the laser coagulation system includes:

a local control system disposed at a first location and a centralcontrol system disposed at a remote site, the remote site being at asecond location;

at least a first laser-imaging system disposed at the first location,the laser-imaging system including a laser-imaging apparatus, a firstprocessor and a local control module;

the laser-imaging apparatus including a digital image acquisition systemconfigured to acquire digital images of the eye structure or the bodysurface, the local control module including local operation, localoperation and performance simulation, and local safety and verificationsub-modules;

a laser generation system configured to generate and transmit at least afirst aiming laser beam and at least a first coagulation laser beam, andmeans for controlling the digital image acquisition system and the lasergeneration system;

a central control system disposed at the remote site, the centralcontrol system including a second processor and a remote control module,the remote control module including remote operation, remote operationand performance simulation, and remote safety and verificationsub-modules; and

the remote operation sub-module being configured to facilitatecommunications between a remote physician and the second processor, andperform a laser coagulation procedure on the eye structure or the bodysurface in an actual control mode, the remote operation sub-moduleincluding a touchscreen interface configured to enable the remotephysician to draw a target laser treatment area or areas on a digitizedimage of the eye structure or the body surface.

In yet another embodiment, the laser coagulation system includes:

a local control system disposed at a first location and a centralcontrol system disposed at a remote site, the remote site being at asecond location, the local control system being operatively coupled tothe central control system by means of a computer network;

at least a first laser-imaging system disposed at the first location,the laser-imaging system including a laser-imaging apparatus, a lasergeneration system, a first computing device with a first processor, anda local control module;

the laser-imaging apparatus including a digital image acquisition systemconfigured to acquire a digitized image of the eye structure or the bodysurface, the local control module including local operation, localoperation and performance simulation, and local safety and verificationsub-modules, the local operation sub-module configured to acquire thedigitized image of the eye structure or the body surface from thedigital image acquisition system and transmit the digitized image to theremote site;

the laser generation system including an aiming laser configured togenerate and transmit an aiming laser beam to the eye structure or thebody surface, and a treatment laser configured to generate and transmitat least a first coagulation laser beam to the eye structure or the bodysurface;

the central control system including a second computing device with asecond processor, and a remote control module, the remote control moduleincluding remote operation, remote operation and performance simulation,and remote safety and verification sub-modules; and

the remote operation sub-module being configured to facilitatecommunications between a remote physician and the second processor ofthe second computing device, and perform a laser coagulation procedureon the eye structure or the body surface in an actual control mode inwhich the treatment laser is configured to transmit the firstcoagulation laser beam to the eye structure or the body surface.

In some embodiments of the invention, the first laser-imaging systemfurther includes an image recognition sensor configured to captureimages of a patient at the first location so that an identity of thepatient or an identity of a body portion of the patient is capable ofbeing identified and verified prior to the laser coagulation procedurebeing performed on the eye structure or the body surface in the actualcontrol mode.

In some embodiments, the image recognition sensor is operatively coupledto the first computing device, the first computing device beingspecially programmed to compare a first reference digital image of thepatient captured by the image recognition sensor at a first time to asecond digital image of the patient captured by the image recognitionsensor at a second subsequent time, and to determine if the seconddigital image of the patient matches or substantially matches the firstreference digital image of the patient (i.e., the second digital imageof the patient substantially matches the first reference digital imageof the patient when there are only minor differences between the twoimages, e.g., a blemish on the face of patient that appears in thesecond digital image, but not in the first reference digital image).

In some embodiments, when the first computing device determines that thesecond digital image of the patient substantially matches the firstreference digital image of the patient, the first computing device isspecially programmed to generate a matched image confirmationnotification that is sent to the second computing device at the remotesite in order to inform the remote physician that the patient has beenidentified and verified; and, when the first computing device determinesthat the second digital image of the patient does not substantiallymatch the first reference digital image of the patient, the firstcomputing device is specially programmed to generate a non-matchingimage notification that is sent to the second computing device at theremote site in order to inform the remote physician that the patient hasnot been properly identified and verified.

In some embodiments, when the first computing device determines that thesecond digital image of the patient does not substantially match thefirst reference digital image of the patient, the first computing deviceis further specially programmed to automatically lock out the treatmentlaser so that the treatment laser is not capable of being fired.

In some embodiments, the image recognition sensor is in the form of amultispectral camera configured to capture the images of the patientusing both visible light and infrared light.

In some embodiments, the first laser-imaging system further includes avoice recognition sensor configured to capture speech waveformsgenerated by the patient at the first location so that an identity ofthe patient is capable of being identified and verified prior to thelaser coagulation procedure being performed on the eye structure or thebody surface in the actual control mode.

In some embodiments, the voice recognition sensor is operatively coupledto the first computing device, the first computing device beingspecially programmed to compare a first reference speech waveform of thepatient captured by the voice recognition sensor at a first time to asecond speech waveform of the patient captured by the voice recognitionsensor at a second subsequent time, and to determine if the secondspeech waveform of the patient matches or substantially matches thefirst reference speech waveform of the patient (i.e., the second speechwaveform of the patient substantially matches the first reference speechwaveform of the patient when there are only minor differences betweenthe two speech waveforms, e.g., a minor difference in the tone of thespeech).

In some embodiments, when the first computing device determines that thesecond speech waveform of the patient matches or substantially matchesthe first reference speech waveform of the patient, the first computingdevice is specially programmed to generate a matched speech confirmationnotification that is sent to the second computing device at the remotesite in order to inform the remote physician that the patient has beenidentified and verified; and, when the first computing device determinesthat the second speech waveform of the patient does not match orsubstantially match the first reference speech waveform of the patient,the first computing device is specially programmed to generate anon-matching speech notification that is sent to the second computingdevice at the remote site in order to inform the remote physician thatthe patient has not been properly identified and verified.

In some embodiments, when the first computing device determines that thesecond speech waveform of the patient does not substantially match thefirst reference speech waveform of the patient, the first computingdevice is further specially programmed to automatically lock out thetreatment laser so that the treatment laser is not capable of beingfired.

In some embodiments, the voice recognition sensor is in the form of amicrophone configured to capture the speech waveforms generated by thepatient over a speech frequency range between 50 Hertz and 5,000 Hertz.

In still another embodiment, the laser treatment system includes:

a local control system disposed at a first location and a centralcontrol system disposed at a remote site, the remote site being at asecond location, the local control system being operatively coupled tothe central control system by means of a computer network;

at least a first laser-imaging system disposed at the first location,the laser-imaging system including a laser-imaging apparatus, a lasergeneration system, a first computing device with a first processor, anda local control module;

the laser-imaging apparatus including a digital image acquisition systemconfigured to acquire a digitized image of the eye structure or the bodysurface, the local control module including local operation, localoperation and performance simulation, and local safety and verificationsub-modules, the local operation sub-module configured to acquire thedigitized image of the eye structure or the body surface from thedigital image acquisition system and transmit the digitized image to theremote site;

the laser generation system including an aiming laser configured togenerate and transmit an aiming laser beam to the eye structure or thebody surface, and a treatment laser configured to generate and transmitat least a first treatment laser beam to the eye structure or the bodysurface;

the central control system including a second computing device with asecond processor, and a remote control module, the remote control moduleincluding remote operation, remote operation and performance simulation,and remote safety and verification sub-modules; and

the remote operation sub-module being configured to facilitatecommunications between a remote physician and the second processor ofthe second computing device, and perform a laser treatment procedure onthe eye structure or the body surface in an actual control mode in whichthe treatment laser is configured to transmit the first treatment laserbeam to the eye structure or the body surface so as to surgically alterthe eye structure or the body surface.

In some embodiments of the invention, the first laser-imaging systemfurther includes an image recognition sensor configured to captureimages of a patient at the first location so that an identity of thepatient or an identity of a body portion of the patient is capable ofbeing identified and verified prior to the laser treatment procedurebeing performed on the eye structure or the body surface in the actualcontrol mode.

In some embodiments, the image recognition sensor is operatively coupledto the first computing device, the first computing device beingspecially programmed to compare a first reference digital image of thepatient captured by the image recognition sensor at a first time to asecond digital image of the patient captured by the image recognitionsensor at a second subsequent time, and to determine if the seconddigital image of the patient substantially matches the first referencedigital image of the patient.

In some embodiments, the image recognition sensor is in the form of aholoscopic three-dimensional camera configured to capture the images ofthe patient in three-dimensional form, and wherein the second computingdevice comprises a graphical user interface in the form of a multiview,three-dimensional visual display device configured to enable the remotephysician or another observer at the remote site to perform athree-dimensional analysis of the three-dimensional images of thepatient that are produced as a result of the patient being instructed toperform a task that alters one or more detectable physical attributes ofthe patient.

In some embodiments, the first computing device is specially programmedto utilize the three-dimensional images so as to collectively take intoaccount a plurality of physiological characteristics of a body area ofthe patient that reflect the one or more altered physical attributeswhen the identity of the patient or the identity of the body portion ofthe patient is identified and verified.

In some embodiments, the multiview, three-dimensional visual displaydevice of the second computing device is in the form of athree-dimensional digital holographic display device.

In some embodiments, the three-dimensional digital holographic displaydevice comprises one or more thin or ultrathin holographic opticalelements for producing high-resolution three-dimensional images, andwherein the three-dimensional digital holographic display devicecomprises an autostereoscopic three-dimensional display to eliminate theneed for the physician or the another observer to wear special eyewearwhile performing the three-dimensional analysis of the three-dimensionalimages of the patient.

In some embodiments, the multiview, three-dimensional visual displaydevice of the second computing device is in the form of a volumetricthree-dimensional display so as to generate the three-dimensional imagesof the patient formed by voxels with spatial depth and volume.

In some embodiments, the second computing device comprises a graphicaluser interface in the form of virtual reality glasses worn by the remotephysician or another observer at the remote site, the virtual realityglasses configured to enable the remote physician or the anotherobserver at the remote site to perform an analysis of the images of thepatient that are produced as a result of the patient being instructed toperform a task that alters one or more detectable physical attributes ofthe patient.

In some embodiments, the laser treatment system further comprises anoptical coherence tomography imaging system, near-infrared opticaltomography imaging system, or a frequency modulated continuous waveimaging system operatively coupled to the first computing device, theoptical coherence tomography imaging system, near-infrared opticaltomography imaging system, or frequency modulated continuous wave systemconfigured to capture additional images of the patient to supplement theimages of the patient captured by the image recognition sensor.

In some embodiments, the image recognition sensor is in the form oftwo-spaced apart cameras configured to capture the images of thepatient, and wherein the second computing device comprises a graphicaluser interface in the form of a head-mounted display device configuredto generate two display images, each of the two display images being infront of a respective one of the right and left eyes of the remotephysician or another observer at the remote site and corresponding tothe images of the patient captured by the two-spaced apart cameras.

In some embodiments, the image recognition sensor is in the form of athree-dimensional multi-color meta-holography device configured tocapture the images of the patient.

In some embodiments, the laser treatment system further comprises aphotoacoustic system being operatively coupled to the first computingdevice, the photoacoustic system including an ultrasound transducerconfigured to detect acoustic waves that are generated as a result ofthe absorption of energy by the eye structure or the body surface suchthat the photoacoustic system is able to capture ultrasonicthree-dimensional images of body structures beneath the skin of thepatient, the body structures beneath the skin of the patient includingbone structures of the patient.

In some embodiments, the treatment laser of the laser generation systemis configured to provide photodynamic therapy to the patient by emittinglight of a predetermined wavelength that is absorbed by tissue of a bodyportion of the patient to which a photosensitizer has been applied, thebody portion of the patient comprising a cancerous tumor, and thephotodynamic therapy configured to treat the cancerous tumor by killingthe cells forming the cancerous tumor.

In some embodiments, the photosensitizer is applied to the tissue of abody portion of the patient comprising the cancerous tumor by using aplurality of nanoparticles, and wherein the light emitted by thetreatment laser of the laser generation system is further absorbed bythe nanoparticles.

In some embodiments, the predetermined wavelength of the light emittedby the treatment laser of the laser generation system is betweenapproximately 380 nanometers and approximately 1550 nanometers.

In some embodiments, the laser treatment system further comprises adisplaceable prism or mirror disposed in the path of the first treatmentlaser beam emitted by the treatment laser, the displaceable prism ormirror being operatively coupled to the first computing device so thatthe displaceable prism or mirror is capable of being selectivelycontrolled by the first computing device based upon instructionsreceived from the second computing device at the remote site from theremote physician, the displaceable prism or mirror configured to enablethe first treatment laser beam to be applied to the tissue of thecancerous tumor of the patient in an oscillatory manner during thephotodynamic therapy.

In some embodiments, the light emitted by the treatment laser of thelaser generation system comprises ultraviolet light, and wherein thepower of the treatment laser is between approximately 2 milliwatts andapproximately 20 milliwatts.

In some embodiments, the digital image acquisition system of thelaser-imaging apparatus is configured to acquire a two-dimensional imageof the tissue of the cancerous tumor of the patient before, during, andafter the photodynamic therapy; and the laser treatment system furthercomprises a photoacoustic system being operatively coupled to the firstcomputing device, the photoacoustic system including an ultrasoundtransducer configured to detect acoustic waves that are generated as aresult of the absorption of energy by the tissue of the cancerous tumorof the patient such that the photoacoustic system is able to captureultrasonic three-dimensional images of the tissue of the cancerous tumorof the patient before, during, and after the photodynamic therapy.

In some embodiments, the photoacoustic system is further configured todetermine a temperature of the tissue of the cancerous tumor of thepatient subjected to laser energy from the first coagulation laser beam,the photoacoustic system further being configured to control the lasergeneration system by maintaining the laser energy of the first treatmentlaser beam at a predetermined energy level so as to prevent exceeding apredetermined threshold temperature during the photodynamic therapy.

In yet another embodiment, the laser treatment system includes:

a local control system disposed at a first location and a centralcontrol system disposed at a remote site, the remote site being at asecond location, the local control system being operatively coupled tothe central control system by means of a computer network;

the local control system including a laser generation device, a firstcomputing device with a first processor, a local control module, and adynamic imaging system;

the laser generation device including a treatment laser configured togenerate and transmit a treatment laser beam to a treatment location onor in the patient;

the central control system including a second computing device with asecond processor and a remote control module;

the remote control module being configured to perform a laser treatmentprocedure on the treatment location on or in the patient in an actualcontrol mode in which the treatment laser is configured to transmit thetreatment laser beam to the treatment location so as to surgically alterthe body of the patient in the treatment location; and

the dynamic imaging system including an imaging device operativelycoupled to the first computing device, the imaging device configured tocapture images of a body portion of the patient over a predeterminedduration of time so that a displacement of the body portion of thepatient is capable of being tracked during the predetermined duration oftime, the first computing device being programmed to determine thedisplacement of the body portion of the patient over the predeterminedduration of time using the captured images, and to compare thedisplacement of the body portion of the patient over the predeterminedduration of time to a reference displacement of the body portion of thepatient acquired prior to the displacement so that dynamic changes inthe body portion of the patient are capable of being assessed for thepurpose of identifying the patient or evaluating physiological changesin the body portion.

In still another embodiment, the telemedicine system includes:

a local control system disposed at a first location and a centralcontrol system disposed at a remote site, the remote site being at asecond location, the local control system being operatively coupled tothe central control system by means of a computer network;

the local control system including a first computing device with a firstprocessor, a local control module, and a dynamic imaging system;

the central control system including a second computing device with asecond processor and a remote control module;

the dynamic imaging system including an imaging device operativelycoupled to the first computing device, the imaging device configured tocapture images of a body portion of the patient over a predeterminedduration of time so that a displacement of the body portion of thepatient is capable of being tracked during the predetermined duration oftime, the first computing device being programmed to determine thedisplacement of the body portion of the patient over the predeterminedduration of time using the captured images, and to compare thedisplacement of the body portion of the patient over the predeterminedduration of time to a reference displacement of the body portion of thepatient acquired prior to the displacement so that dynamic changes inthe body portion of the patient are capable of being assessed for thepurpose of identifying the patient or evaluating physiological changesin the body portion.

In yet another embodiment, the telemedicine system includes:

a local control system disposed at a first location and a centralcontrol system disposed at a remote site, the remote site being at asecond location, the local control system being operatively coupled tothe central control system by means of a computer network;

the local control system including a first computing device with a firstprocessor, a local control module, and a facial recognition system;

the central control system including a second computing device with asecond processor and a remote control module;

the facial recognition system including an imaging device operativelycoupled to the first computing device, the imaging device configured tocapture one or more images of a face of the patient, the first computingdevice being programmed to compare one or more first reference digitalimages of the face of the patient captured by the imaging device offacial recognition system at a first time to one or more second digitalimages of the face of the patient captured by the imaging device of thefacial recognition system at a second subsequent time, and to determineif the one or more second digital images of the face of the patientsubstantially matches the one or more first reference digital images ofthe face of the patient.

In still another embodiment, the laser coagulation system includes:

a local control system disposed at a first location and a centralcontrol system disposed at a remote site, the remote site being at asecond location, the local control system being operatively coupled tothe central control system by means of a computer network;

at least a first laser-imaging system disposed at the first location,the laser-imaging system including a laser-imaging apparatus, a lasergeneration system, a first computing device with a first processor, anda local control module;

the laser-imaging apparatus including a digital image acquisition andstorage system configured to take and store digital images of the eyestructure or the body surface, the digital image acquisition and storagesystem further including means for transmitting digital images;

the laser generation system including an aiming laser configured togenerate and transmit an aiming laser beam to the eye structure or thebody surface, and a treatment laser configured to generate and transmitat least a first coagulation laser beam to the eye structure or the bodysurface, and means for controlling the digital image acquisition andstorage system and the laser generation system;

the local control module including a local operation sub-module, a localoperation and performance simulation sub-module, the local operationsub-module configured to acquire at least a first digitized image of theeye structure or the body surface from the digital image acquisition andstorage system and transmit the first digitized image to the remotesite, the local operation sub-module further configured to receive atarget laser transmission area or areas and laser transmissionparameters from a remote physician, and determine a treatment area orpattern of spots on the first digitized image for application of thefirst coagulation laser beam, the local operation sub-moduleadditionally configured to perform a laser coagulation of the eyestructure or the body surface under the remote physician's command inwhich the first coagulation laser beam is applied to the treatment areaor each of the spots in the pattern, the local operation and performancesimulation sub-module configured to facilitate the testing of the systemprior to its operation in an actual control mode by replacing an actualeye structure or body surface of the subject with the first digitizedimage of the subject's eye structure or body surface;

the central control system being in communication with the laser-imagingapparatus, the central control system including a remote control moduleand a second computing device with a second processor configured toreceive and process command signals from the remote physician andtransmit the command signals to the local control module; and

the remote control module including a remote operation sub-moduleconfigured to facilitate communications between the remote physician andthe second processor of the second computing device, the remoteoperation sub-module, in association with the local operation andperformance simulation sub-module, further configured to execute avirtual treatment of the eye structure or the body surface, perform atest surgical procedure, and perform a fully automated and continuouslaser coagulation procedure over the entire area of the eye structure orthe body surface in the actual control mode in which the treatment laseris configured to transmit the first coagulation laser beam to the eyestructure or the body surface

In yet another embodiment, the laser coagulation system includes:

a local control system disposed at a first location and a centralcontrol system disposed at a remote site, the remote site being at asecond location;

at least a first laser-imaging system, the laser-imaging systemincluding a laser-imaging apparatus, a first processor and a localcontrol module;

the laser-imaging apparatus including a wide angle digital imageacquisition and storage system configured to take and store digitalimages of the retina, the digital image acquisition and storage systemincluding at least one retinal viewing camera that provides a field ofview of the retina in a range between 160° and 200°, the digital imageacquisition and storage system further including means for transmittingdigital images;

a laser generation system configured to generate and transmit at least afirst aiming laser beam and at least a first coagulation laser beam, andmeans for controlling the digital image acquisition and storage systemand the laser generation system;

the local control module including a local operation sub-module, a localoperation and performance simulation sub-module, and a local safety andverification sub-module, the local operation sub-module configured toacquire at least a first retinal image of the retina from the digitalimage acquisition and storage system and transmit the first retinalimage to the remote site, the local operation sub-module furtherconfigured to receive a target laser transmission area and lasertransmission parameters from a remote physician, and determine a patternof spots on the first retinal image for application of the firstcoagulation laser beam, the local operation sub-module additionallyconfigured to perform a scatter laser coagulation of the retina underthe remote physician's command in which the first coagulation laser beamis applied to each of the spots in the pattern, acquire a plurality ofpost-procedure retinal images, and transmit the post-procedure retinalimages to the remote site for evaluation and verification of treatment,the local operation and performance simulation sub-module configured tofacilitate the testing of the system prior to its operation in an actualcontrol mode by replacing an actual eye of the subject with a digitizedfundus image of the subject's eye;

the central control system being in communication with the laser-imagingapparatus, and including a remote control module and a second processorconfigured to receive and process command signals from the remotephysician and transmit the command signals to the local control module;

the remote control module including a remote operation sub-moduleconfigured to facilitate communications between the remote physician andthe second processor, execute a virtual treatment of the retina, performa test surgical procedure, and perform a fully automated and continuouslaser coagulation procedure over the entire area of the retina in theactual control mode;

the remote control module further including a remote operation andperformance simulation sub-module and a remote safety and verificationsub-module, the remote operation and performance simulation sub-moduleconfigured to test performance parameters of the local operation moduleand perform a treatment simulation of the retina by the remote physicianwhile simulating eye movement of the subject by displacing the digitizedfundus image of the subject's eye in accordance with a plurality ofrandom variables;

the local and remote safety and verification sub-modules includingphysical, logical and medical safety constraints for safe operation ofthe system; and

wherein the system for laser coagulation of the retina is in the form ofa non-contact system that does not require the use of a contact lens.

BRIEF DESCRIPTION OF THE DRAWINGS

Further features and advantages will become apparent from the followingand more particular description of the preferred embodiments of theinvention, as illustrated in the accompanying drawings, and in whichlike referenced characters generally refer to the same parts or elementsthroughout the views, and in which:

FIG. 1 is an illustration of a human eye showing the major structuresthereof;

FIG. 2 is a schematic illustration of one embodiment of a laser-imagingapparatus, in accordance with the invention;

FIG. 3 is a schematic illustration of another embodiment of alaser-imaging apparatus, showing the elliptical mirror thereof, inaccordance with the invention;

FIG. 4 is a schematic illustration of a photoacoustic system, inaccordance with the invention;

FIG. 5 is a schematic illustration of one embodiment of a laser-imagingsystem, in accordance with the invention;

FIG. 6 is another schematic illustration of the laser-imaging systemshown in FIG. 5, showing the local and remote modules thereof, inaccordance with one embodiment of the invention;

FIG. 7 is an illustration of a human retina, showing an oval areaencompassing the target area on the retina for laser transmission, inaccordance with one embodiment of the invention;

FIG. 8 is an illustration of a human retina, showing the distribution oflaser spots resulting from a virtual treatment of the retina, inaccordance with one embodiment of the invention;

FIG. 9 is an illustration of a human retina, showing simulationvariables associated with movement of the eye, in accordance with oneembodiment of the invention;

FIG. 10 is an illustration of a human retina, showing a generated gridmap on the retina, in accordance with one embodiment of the invention;

FIG. 11 is an illustration of a human retina, showing a large area ofthe fundus that is to be coagulated, in accordance with one embodimentof the invention;

FIG. 12 is an illustration of a human retina, showing localized areas ofthe fundus that are to be coagulated, in accordance with one embodimentof the invention;

FIG. 13 is an illustration of a human retina, showing single, localizedspots of the fundus that are to be coagulated, in accordance with oneembodiment of the invention;

FIG. 14 is an illustration of a human face, showing an area of a skinlesion that is marked for laser application;

FIG. 15 is a schematic illustration of another embodiment of alaser-imaging system, in accordance with the invention, wherein thelocal control system of the laser-imaging system comprises image andvoice recognition sensors;

FIG. 16 is an illustration of a human face, showing exemplary areas ofthe human face that may be imaged so that the identity of a patient orthe identity of a body portion of a patient may be verified;

FIG. 17 is a schematic illustration of another embodiment of alaser-imaging system, in accordance with the invention, wherein theremote laser treatment system is provided with GPS-based identificationof the laser treatment application site;

FIG. 18a is an illustration of a human face, depicting a grid that isprojected over the facial area being analyzed using the dynamic imagingsystem of the remote laser treatment system described herein, andfurther depicting the positions of two exemplary points being used totrack dynamic changes in the lips of the person;

FIG. 18b is another illustration of a human face, depicting a grid thatis projected over the facial area being analyzed using the dynamicimaging system of the remote laser treatment system described herein,and further depicting the positions of two exemplary points being usedto track dynamic changes in the lips of the person while the person isspeaking the letter “O”;

FIG. 19a is an illustration of a finger being pressed against atransparent glass surface so that the tip of the finger is capable ofbeing imaged;

FIG. 19b depicts a finger before touching a transparent glass surfaceused for the imaging of the finger;

FIG. 19c depicts the finger touching the transparent glass surface, thefinger undergoing imaging that takes into account both surface andsubsurface properties of the finger in a two-dimensional and/orthree-dimensional manner;

FIG. 20 depicts an initial image of a face of a person captured by adynamic identity recognition system where the person has hair and facialhair;

FIG. 21 depicts another image of the face of the person in FIG. 20captured by the dynamic identity recognition system where the appearanceof the midsection of the face has remained generally stable, but dynamicchanges have occurred in the lower section of the face;

FIG. 22 depicts a computer-modified image of the face of the person inFIG. 20 where the hair and facial hair of the person have been removed,and the stable features of the person are depicted;

FIG. 23 depicts another image of the face of the person in FIG. 20captured by the dynamic identity recognition system where the appearanceof the midsection of the face has remained generally stable, but dynamicchanges have occurred in the lower section of the face;

FIG. 24 depicts a computer-modified image of the face of the person inFIG. 20 where the hair and facial hair of the person have been removed,and glasses and an earring have been added to the person;

FIG. 25 depicts a computer-modified image of the face of the person inFIG. 20 where an artificial change in the appearance of the person hasoccurred while the person is speaking one or more words; and

FIG. 26 depicts a graphical representation of a correlation betweenmouth configuration and a sound wave generated by a person whilespeaking a series of words.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT

Before describing the present invention in detail, it is to beunderstood that this invention is not limited to particularlyexemplified apparatus, systems, structures or methods as such may, ofcourse, vary. Thus, although a number of apparatus, systems and methodssimilar or equivalent to those described herein can be used in thepractice of the present invention, the preferred apparatus, systems,structures and methods are described herein.

It is also to be understood that the terminology used herein is for thepurpose of describing particular embodiments of the invention only andis not intended to be limiting.

Unless defined otherwise, all technical and scientific terms used hereinhave the same meaning as commonly understood by one having ordinaryskill in the art to which the invention pertains.

Further, all publications, patents and patent applications cited herein,whether supra or infra, are hereby incorporated by reference in theirentirety.

Finally, as used in this specification and the appended claims, thesingular forms “a, “an” and “the” include plural referents unless thecontent clearly dictates otherwise. Thus, for example, reference to “alaser image” includes two or more such images and the like.

Definitions

The terms “eye disorder” and “eye disease” are used interchangeablyherein and mean and include, without limitation, diabetic retinopathy,vascular occlusion, neovascularization, retinal detachment, neoplastictissue, ischemic retina, retinopathy of prematurity and age relatedmacular degeneration.

The following disclosure is provided to further explain in an enablingfashion the best modes of performing one or more embodiments of thepresent invention. The disclosure is further offered to enhance anunderstanding and appreciation for the inventive principles andadvantages thereof, rather than to limit in any manner the invention.The invention is defined solely by the appended claims including anyamendments made during the pendency of this application and allequivalents of those claims as issued.

As will readily be appreciated by one having ordinary skill in the art,the present invention substantially reduces or eliminates thedisadvantages and drawbacks associated with conventional systems andmethods for coagulating eye structures to treat eye disorders.

In overview, the present disclosure is directed to laser imaging andcoagulation apparatus, systems and methods that allow an eye specialist,e.g., an ophthalmologist or surgeon, to perform laser retinal surgicalprocedures, such as laser or tissue coagulation, with an integral laserimaging and coagulation apparatus disposed at a first (i.e. local)location from a control system disposed at a second (i.e. remote)location, e.g., a physician's office.

By the term “laser coagulation”, as used herein, it is meant to mean andinclude, without limitation, selective absorbance of transmitted lightenergy (having a visible green wavelength) by hemoglobin in biologicaltissue and, hence, sealing of blood vessels in the tissue. In apreferred embodiment of the invention, the wavelength of the transmittedenergy (referred to herein as a “treatment or coagulative laser beam”)is in the range of approximately 400-1650 nm, more preferably, in therange of approximately 530-670 nm.

Although the present invention is directed to thermotherapy ofbiological tissue by laser energy, it is to be understood the inventionis not limited to such form of energy. Indeed, as will readily beappreciated by one having ordinary skill in the art, the thermotherapyof biological tissue described herein, i.e. coagulation of selective eyestructures, can also be achieved via the application of electromagneticradiation, and radio frequency and ultrasound energy.

It is further to be understood that, although the biological tissuesubject to the thermotherapy (i.e. coagulation), in accordance with thepresent invention, comprises the retina, the invention is not limitedsolely to thermotherapy of the retina. According to the invention, thethermotherapy of the present invention can be employed to coagulate anyeye structure.

The laser-imaging apparatus, systems and methods of the invention, andlaser energy transmitted thereby, can thus be employed to treat variouseye disorders, including, without limitation, diabetic retinopathy,vascular occlusion, neovascularization, retinal detachment, neoplastictissue, ischemic retina, retinopathy of prematurity and age relatedmacular degeneration.

The laser-imaging apparatus, systems and methods of the invention, andlaser energy transmitted thereby, can also be readily employed inrefractive surgical procedures to, for example, perform corneal surfaceablation using an eximer or femtosecond laser, LASIK procedures, and/orlid surface and surrounding tissue tightening using an infrared laser.

The laser-imaging apparatus, systems and methods of the invention, andlaser energy transmitted thereby, can also be readily employed incosmetic surgical procedures to, for example, remove skin lesions andperform skin resurfacing.

Before describing the invention in detail, the following briefdescription of the various anatomical features of the eye is provided,which will help in the understanding of the various features of theinvention:

Referring to FIG. 1, the cornea 10, which is the transparent window thatcovers the front of the eye 100, is a lens-like structure that providestwo-thirds of the focusing power of the eye.

The cornea 10 is slightly oval, having an average diameter of about 12mm horizontally and 11 mm vertically. The central thickness of thecornea 10 is approximately 550 μm.

The sclera 16 is the white region of the eye, i.e. posterior five sixthsof the globe. It is the tough, avascular, outer fibrous layer of the eyethat forms a protective envelope. The sclera is mostly composed of densecollagen fibrils that are irregular in size and arrangement (as opposedto the cornea). The extraocular muscles insert into the sclera behindthe limbus.

The sclera 16 can be subdivided into 3 layers: the episclera, scleraproper and lamina fusca. The episclera is the most external layer. It isa loose connective tissue adjacent to the periorbital fat and is wellvascularized.

The sclera proper, also called tenon's capsule, is the layer that givesthe eye 100 its toughness. The sclera proper is avascular and composedof dense type I and III collagen.

The lamina fusca is the inner aspect of the sclera 16. It is locatedadjacent to the choroid and contains thin collagen fibers and pigmentcells.

The pars plana is a discrete area of the sclera 16. This area is avirtually concentric ring that is located between 2 mm and 4 mm awayfrom the cornea 10.

The vitreous humor or vitreous 12 is the largest chamber of the eye 100(i.e. ˜4.5 ml). The vitreous 12 is a viscous transparent gel composedmostly of water. Unlike the fluid contained in the frontal parts of theeye (e.g., aqueous humor, discussed below), which are continuouslyreplenished, the transparent gel in the vitreous chamber is stagnant.

As is well known in the art, the vitreous humor 12 also contains arandom network of thin collagen fibers, mucopolysaccharides andhyaluronic acid.

The aqueous humor 14 occupies the anterior chamber 18 of the eye 100.The aqueous humor 14 has a volume of about 0.6 mL and provides nutrientsto the cornea 10 and lens 28.

One of the most important functions of the aqueous humor 14 is tomaintain IOP by the rate of its production and drainage.

The additional parts of the eye that are illustrated in FIG. 1 comprisethe uvea, and structures thereof, lens 28 and retina 30.

The uvea refers to the pigmented layer of the eye 100 and is made up ofthree distinct structures: the iris 22, ciliary body, and choroid 24.The iris 22 is the annular skirt of tissue in the anterior chamber 18that functions as an aperture. The pupil is the central opening in theiris 22.

The ciliary body is the 6 mm portion of uvea between the iris 22 andchoroid 24. The ciliary body is attached to the sclera 16 at the scleralspur. It is composed of two zones: the anterior 2 mm pars plicata, whichcontains the ciliary muscle 26, vessels, and processes, and theposterior 4 mm pars plana.

The ciliary muscle 26 controls accommodation (focusing) of the lens 28,while the ciliary processes suspend the lens 28 (from small fibers, i.e.zonules) and produce the aqueous humor 14 (the fluid that fills theanterior and posterior chambers and maintains intraocular pressure).

The choroid 24 is the tissue disposed between the sclera 16 and retina30. The choroid 24 is attached to the sclera 16 at the optic nerve 20and scleral spur. This highly vascular tissue supplies nutrients to theretinal pigment epithelium (RPE) and outer retinal layers.

The layers of the choroid 24 (from inner to outer) include the Bruch'smembrane, choriocapillaris and stroma. Bruch's membrane separates theRPE from the choroid 24 and is a permeable layer composed of thebasement membrane of each, with collagen and elastic tissues in themiddle.

The crystalline lens 28, located between the posterior chamber and thevitreous cavity, separates the anterior and posterior segments of theeye 100. Zonular fibers suspend the lens from the ciliary body andenable the ciliary muscle to focus the lens 28 by changing its shape.

The retina 30 is the delicate transparent light sensing inner layer ofthe eye 100. The retina 30 faces the vitreous and consists of two basiclayers: the neural retina and retinal pigment epithelium. The neuralretina is the inner layer. The retinal pigment epithelium is the outerlayer that rests on Bruch's membrane and choroid 24.

As indicated above, conventional slit lamp systems, which are oftenemployed to treat various eye disorders, such as diabetic retinopathy,require a contact lens to neutralize the refractive power of the corneaand to provide a variable field of view of the retina.

The length of time to perform a surgical procedure with a conventionalslit lamp system is also presently in the range of 30 minutes to anhour. There is thus an increased percentage of probable error due to thelaser photo-coagulation being controlled manually, i.e. by thephysician's hand, and the potential eye movements from the patientduring this extended period of time.

The present invention substantially reduces or eliminates thedisadvantages and drawbacks associated with conventional slit lampsystems and associated methods. As discussed in detail herein, thelaser-imaging apparatus include means for taking and storing digitalimages of the target eye structure(s), which can be retrieved on amonitor for diagnosis and defining the area of treatment. In someembodiments, the laser-imaging apparatus (and systems) of the inventionthus include a retinal camera (e.g., Topcon, Zeiss, Kowa or preferably awide angle viewing system having an elliptical mirror), which, in someembodiments, is modified with a wide-angle lens.

A wide field scanning laser-imaging apparatus, such as the laseropthalmoscope disclosed in U.S. Pat. No. 5,815,242, can also be employedto provide images of the fundus, particularly, the retina. The notedlaser-imaging apparatus can also be readily modified for lasercoagulation procedures in one or multiple health care applications oradditional vision care offices.

According to the invention, the viewing light can comprise a white lightfrom a flush light, laser source or one or more scanning lasers withcompensatory wavelengths in the range of approximately 190 nm-10,000 nm,more preferably, in the range of approximately 400-1060 nm, to obtain afundus photograph.

According to the invention, the retinal camera is connected to lasertransmission means (or a laser system) that is adapted to generate andtransmit laser energy that is sufficient to coagulate any desiredportion of the retina using a monitor's touch screen.

In a preferred embodiment of the invention, the laser-imaging apparatusincorporates aiming and treatment laser generating and transmissionmeans into an existing camera (e.g., Topcon, Zeiss, etc.) or a wideangle viewing system/camera, such as the laser opthalmoscope disclosedin U.S. Pat. No. 5,815,242. The laser-imaging apparatus also includescontrol means for controlling the aiming and treatment laser means, andthe camera.

Preferably, the transmitted laser beam passes through the optical pathof the viewing apparatus (or system) and is preferably reflected off ofan elliptical mirror in the camera (in which the imaging light isfocused toward the pupil or slightly behind the pupil), providing afield of view greater than approximately 200°.

Referring now to FIG. 2, there is shown one embodiment of alaser-imaging apparatus of the invention. As illustrated in FIG. 2, thelaser-imaging apparatus 200 includes a Topcon digital camera 202,scanning laser visualization means 204 and internal laser generation andtransmission means (i.e. coagulation means) 206. The laser-imagingapparatus 200 further includes a refracting lens 203, at least onetwo-way mirror 208 and a plurality of appropriately positionedreflecting mirrors 201 a-201 c.

Referring now to FIG. 3, in an alternative embodiment, a wide anglecamera equipped with an elliptical mirror 220 is employed. According tothe invention, the illustrated wide angle camera provides an improvedrange of between approximately 150° and approximately 200°, inclusive,(or an improved range between 150° and 200°, inclusive) of the retinafor optimal treatment.

In some embodiments, the concave, elliptical mirror 220 illustrated inFIG. 3 is configured to oscillate (or wobble) slightly in order to shiftthe focal point of the mirror 220 slightly from one side of the pupil tothe other side, thereby permitting the scanning light (e.g., lowcoherent wavelengths, etc.) inside the eye to cover a larger peripheralfield than possible without oscillation. An exemplary imaging system forimaging the central and peripheral retina, which employs such anoscillating concave mirror, is disclosed in Applicant's U.S. Pat. No.8,070,289; which is incorporated by reference herein in its entirety.

In other embodiments, the concave, elliptical mirror 220 illustrated inFIG. 3 is stationary and is not configured to oscillate or wobble. It isalso to be understood that the concave mirror 220 can be provided in theform of circular mirror, as well as an elliptical mirror.

FIG. 3 shows the path of the viewing/imaging scanning laser beams 214 asthey are reflected and pass through the focal point of the mirror behindthe pupil (Black) toward the retina. As illustrated in FIG. 3, thecoagulative laser beam 216 preferably passes through the same path asthe viewing/imaging beams 214.

In some embodiments of the invention, the laser-imaging apparatus of theinvention also includes an optical coherence tomography (OCT) means.

Preferably, the rays reflected back from the retina pass through thesame pathway and form a digital image that can be observed on themonitor.

According to the invention, the coagulative laser beam is also scannedover the retinal area via the same light path as used for theobservation and documentation.

The integration of the aiming and treatment laser generating andtransmission means with a camera requires the introduction of precisionmotorized optical fixtures. An opto-mechanical system having an integralcontrol system is thus provided to control and/or position the targetspot of the laser beam(s) in x and y directions within the eye. Thesystem is designed and adapted to interface with joystick commandsand/or computer/monitor touch screen commands, for local and remotecontrol of the aiming and treatment (or coagulative) lasertransmissions, respectively.

In some embodiments of the invention, the control means for positioningthe laser transmissions (or beams) within the eye consists of two maincomponents. The first component is adapted to move the beams in thex-direction. The second component is adapted to move the beams in they-direction.

Preferably, movement in the y-direction is provided by a mirroredsurface disposed in the optical path of the camera. This y-direction,motorized fixture provides precise movement of the mirrored surface,while still allowing diagnostic and treatment images to be seen throughthe retinal camera.

In some embodiments of the invention, producing the x-direction movementinvolves physically moving the laser unit; the movement of the laserbeing either translational or rotational. Various conventional means ofmotorized movement can also be employed to provide movement in thex-direction.

In a preferred embodiment, the laser-imaging apparatus is incommunication with another remote system via the Internet®, whereby thelaser-imaging apparatus can be controlled by a physician at the remotesite (e.g., medical center).

According to the invention, location of laser energy or beam applicationcan be from 5-200° of the retina. In some embodiments, location of laserenergy application is preferably 30-200° of the retina.

In some embodiments of the invention, the transmitted coagulative laserenergy (or beam(s)) has a wavelength in the range of approximately400-1650 nm, more preferably, in the range of approximately 530-670 nm.The laser (or laser energy) can also be transmitted in a pulsed manneror continuously.

According to the invention, the laser spot size can be in the range ofapproximately 10 micron-1500 micron.

According to the invention, exposure time of the laser energyapplication can be in the range of approximately 1 femto-seconds to 1000seconds.

In some embodiments of the invention, the laser-imaging apparatus 200includes a photoacoustic system that can measure the temperature insidethe eye tissue during and after laser scanning. A preferredphotoacoustic system is disclosed in Applicant's U.S. Pat. No.8,121,663; which is incorporated by reference herein in its entirety.

As set forth in detail in the '663 patent, the photoacoustic system isadapted to record the sonic waves that are generated by heating eyetissue, e.g. retina tissue. This provides precise information of thetemperature generated as a result of the laser transmission, i.e.coagulative laser energy.

Advantageously, the photoacoustic system enables the temperaturegenerated at the treatment site to be measured. As a result, the systemis capable of balancing the energy of the laser system so that thecoagulation is performed in a uniform fashion at the desired area,without such balancing one could have some lesions stronger than othersdepending on the degree of the pigmentation of the retina at theparticular site (i.e., if the site absorbs more laser light).

Referring now to FIG. 4, there is shown one embodiment of aphotoacoustic system 80. As illustrated in FIG. 4, the photoacousticsystem 80 includes a laser source 88, an ultrasonic detector 89, and aprobe module 82. The probe module 82 includes an objective lensstructure 84, which is preferably coupled to the light source 88 via afiber optic connection or other light transmitter. Alternatively, thelight source can be incorporated into the probe module 82.

According to the invention, the light source 88 can comprise a laser,laser diode or superluminescent diode (SLD), as appropriate forgenerating the desired light wavelength and intensity. The light canalso be delivered as pulses or as modulated radiation.

As further illustrated in FIG. 4, the probe module 82 further containsan ultrasound transducer 86 that is adapted to detect the photoacousticwaves that are generated as a result of the absorption of energy fromthe light emitted by the objective lens structure 84. The ultrasoundtransducer 86 is in contact with the eye 100 or an eyelid drawn over theeye.

As light is delivered as pulses or as modulated radiation, pulses ormodulating acoustic signals are generated and returned to the ultrasoundtransducer 86 in probe module 82.

According to the invention, localization of the source of photoacousticsignals can be achieved in various manners. First, localization can beaccomplished by directing the beam from objective lens structure 84 inspecific directions, by moving that structure with micromechanicalactuators, as shown diagrammatically at 85 in FIG. 4, thus targeting aparticular line of points in the eye.

Furthermore, by suitable optics included in objective lens structure 84,the focal point of the emitted light may be moved within the eye to adesired point, such as a point along the retina vasculature, toselectively generate acoustic signals at that desired point. Because theeye is optically transmissive relative to soft tissue, beam focusing andbeam directing are likely to be more accurately performed in the eye,than is the case is soft tissue elsewhere in the body.

To further assist in directionally capturing the photoacoustic signalsgenerated within the eye, a directional transducer array can be used astransducer 86, to control the directionality of reception of ultrasonicenergy, thus further localizing upon a desired source of thermoacousticsignals. Thus, by targeting the focal point of the illuminating light,and also directionally targeting the reception of ultrasonic signals bythe transducer array, thermoacoustic signals from a particular location,such as along the retina, may be specifically targeted.

Overview of Laser-Imaging System

Referring now to FIG. 5, there is shown a schematic diagram of alaser-imaging system 50, according to one embodiment of the invention.As illustrated in FIG. 5, the system 50 includes at least one,preferably, multiple local systems 52, 54, 56 disposed at local sites 51a, 51 b, 51 c; each system 52, 54, 56 including a laser-imagingapparatus (denoted “53”, “55” and “57”), such as the apparatus 200 shownin FIG. 2. In a preferred embodiment of the invention, eachlaser-imaging apparatus 53, 55, 57 includes a photoacoustic system, suchas system 80 discussed above.

Preferably, each laser-imaging apparatus 53, 55, 57 is preferably incommunication with a local control module 62 and control-processingmeans 59 a, such as a personal computer. Each of the modules describedherein in conjunction with the embodiments of the laser-imaging systems50, 50′ may comprise one or more hardware components together with therequisite software to carry out the functionality performed thereby.

In some embodiments of the invention, at least the control-processingmeans 59 a disposed at each local site 51 a, 51 b, 51 c includes facialrecognition means for identifying and/or verifying the identity of asubject or patient. Alternatively, in some embodiments, thecontrol-processing means 59 b disposed at the remote site 51 d(discussed below) includes facial recognition means. In some embodimentsboth control-processing means 59 a, 59 b include facial recognitionmeans.

In some embodiments of the invention, each local system 52, 54, 56 alsoincludes eye tracking means 71 for measuring eye position(s) andmovement. According to the invention, the eye tracking means 71 can bean integral component or feature of the laser-imaging apparatus 53, 55,57 or a separate system or device.

Also disposed at each local site 51 a, 51 b, 51 c during a laserprocedure is a test subject or patient and a physician or technician.

As also illustrated in FIG. 5, the system 50 also includes a remote site58 having a command computer 59 b that is operatively connected to aremote control module 64. Also disposed at the remote site 58 during alaser procedure is a system operator (e.g., retinal surgeon).

As discussed in detail below, communication by and between the localsites 52, 54, 56 and the remote site 58 is preferably facilitated by thelocal and remote control modules 62, 64 and the Internet® 60.

In accordance with one embodiment of the invention, the sequence ofinteractions between the local sites 52, 54, 56 and remote site 58comprises the following:

-   -   Fundus photograph is digitally transmitted to remote site;    -   Image is acquired by remote site and filed in the computer at        remote site;    -   Eye is repositioned in front of the camera, fundus image is        taken for pattern recognition and tracking, and transmitted to        remote site, verified as matching previous, pre-existing image        on file;    -   New image is selected and employed as simulator;    -   Spot size, power level, and time interval between each laser is        chosen;    -   Tracking system (fail-safe) is checked;    -   Fundus laser treatment is performed in virtual mode to establish        the desired laser coagulation; and    -   After choosing spot size and duration of laser application,        power is adjusted to the lowest increment of energy that may or        may not be able to create a response on the patient's eye. The        power of the laser is incrementally increased until the test        spot demonstrates the desired effect.

In a preferred embodiment, the optimum power and, hence, temperaturerequired for a procedure is provided via the photoacoustic system. Thephotoacoustic system is further adapted to maintain the transmittedlaser energy at a fixed, pre-selected level. This particularly resultsin a uniform coagulation of the retina.

Local and Remote Control Modules

Referring now to FIG. 6, there is shown a schematic diagram of a localcontrol module 62 and a remote control module 64, according to oneembodiment of the invention. As illustrated in FIG. 6, the local controlmodule 62 preferably includes three sub-modules: an operation module 65,safety and verification module 66, and an operation and performancesimulation module 67.

The remote control module 64 similarly includes three sub-modules: anoperation module 70, safety and verification module 72, and an operationand performance simulation module 74.

Each of these sub-modules 65, 66, 67 is described below.

Local Operation Module

According to the invention, the local operation module 65 provides alocal technician with an interface to a personal computer for dataacquisition and eye treatment.

According to the invention, the major tasks performed by the localoperation module 65 include the following:

(1) acquiring a plurality (preferably, in the range of 5-7) standardfields of the fundus retinal images and transmission of the images tothe remote site 58;

(2) receiving the oval area encompassing the focused region of theretina outlined by the remote physician (see FIGS. 5 and 7), as well asparameters for the spot size, power and duration of laser application;

(3) applying a snake algorithm (i.e. active contour algorithm) toextract the contours, calculating the corresponding areas between thecontours, and partitioning the image into a grid map (see FIG. 10), as afunction of the specified surgical area and corresponding parameters forlaser application from the remote center. According to the invention,the resolution of the grid is adjusted according to the area between thecontours, whereby the number of nodes between the contours (i.e. theblack points between the two white contours) is large enough to generatein the range of approximately 700-1000 laser spots to ensure the surgeryprecision;

(4) performing the scatter laser coagulation under the remote doctor'scommand. During the process (which is performed in real-time), theKalman filter and mean shift algorithm are preferably integrated todetect and estimate the test subject's eye movement: a) if the movementis under the given safety threshold, the laser system will be adjustedusing a simple proportional-integral-derivative (PID) control algorithmbased on the estimated motion so that it will be focused on the remotelyspecified spot within allowable surgical accuracy, and then the laserwill be applied; b) if the estimated movement is beyond the specifiedsafety threshold, the laser coagulation procedure will be terminatedimmediately and a warning message will be transmitted to the remotecontrol module 64. Step (1), above, will also be repeated until a stopinstruction is received from the remote physician; and

(5) acquiring a plurality (preferably, in the range of 4-5) standardfields of fundus retinal images and transmitting the images to theremote site for evaluation and verification of treatment.

Local Operation and Performance Simulation Module

According to the invention, the local operation and performancesimulation module 67 allows the physician (or technician) to test theentire system 50 (e.g., tracking and interruption functions in the localoperation module 65; communications between local and remote sites;safety and verification modules 66, 72 at both remote and local sites)before the system 50 is run in an actual control mode.

In the simulation mode, the local simulation module 67 replaces the testsubject (or patient), but all the other modules (e.g., modules at theremote site 70, 72, 74; safety and verification module 66 and localoperation module 65 at the local site) preferably operate in exactly thesame manner.

In one or more embodiments of the invention, the local operation andperformance simulation module 67 is configured to test the trackingfunctions of the local operation module 65 by replacing the test subjectwith a digitized fundus image of him or her. The tracking system istested using the digitized fundus image in place of the test subject bydisplacing the digitized fundus image in the X, Y, and O directions (seeFIG. 9). The displacement of the digitized fundus image simulates thehead and/or eye movement of the test subject that is experienced in theactual control mode. The laser is activated in the simulation mode, butis configured to turn off if the movement of the digitized fundus imageexceeds a certain predetermined threshold so as to simulate the actualmovement of the test subject exceeding a predetermined threshold.

Any rapid displacement beyond a certain predetermined threshold value orrange of values (e.g., an orientation change exceeding 3-5 degrees)cannot be immediately compensated for by the tracking system. As aresult, in such a situation, the local operation and performancesimulation module 67 is configured to simulate the response of thetracking system to the detection of a displacement exceeding thethreshold value or range by shutting down the laser of the lasercoagulation system. In the simulation mode, the digitized fundus imagecan be slightly tilted or laterally displaced (e.g., moving the image onthe screen of a visual display device or mechanically, by displacing thescreen itself containing the image with a multi-dimensional actuator) tosimulate the deactivation of the laser. In some embodiments, thetracking system is configured to follow a particular spot on the fundusimage. When the spot being followed by the tracking system is rapidlytilted or displaced, the laser is shut down by the laser coagulationsystem. This simulated test is used to ensure that the tracking systemis fully operational and functioning properly prior to the performanceof the laser surgery (i.e., in the actual control mode).

While performing the simulation using a digitized fundus image of theeye is preferred, a physical model of an eye can also be used to performthe simulation carried out by the local operation and performancesimulation module 67. For example, an artificial eye can be placed infront of the lens of the laser-imaging apparatus. In such an artificialeye, the retina of the eye is visible through the pupil thereof. In thisconfiguration, the tracking system of the laser coagulation system istested by slightly displacing the artificial eye in front of the lens ofthe laser-imaging apparatus (e.g., by using mechanical means, such as amulti-dimensional mechanical actuator) while the laser is activated, butis not firing any actual laser shots. When the detected displacementexceeds the threshold value or range, the tracking system is triggeredso as to shut down the laser.

Remote Operations Module

According to the invention, the remote operation module 70 provides thephysician with an interface to a personal computer. During a laserprocedure, some of the important tasks that the physician will performand are facilitated by the remote operation module) include: (1)screening via digitized photos, (2) outlining an oval area encompassingthe center part of the retina (see FIGS. 5 and 7), (3) choosing the spotsize and duration of laser application, (4) adjusting the power viasingle shots of test spots (preferably, with a duration in the range ofapproximately 0.001-0.1 seconds in approximately 0.06 intervals), (5)executing virtual treatment with the simulation module (see FIG. 6), (6)performing a test surgery, which involves the local operation andperformance simulation module 67, and (7) performing the laser surgeryin the actual control mode (and observing the surgery via a real-timevideo stream).

In one or more embodiments of the invention, the remote operation module70 further comprises an electronic visual display device withtouchscreen capabilities, which is operatively coupled to a personalcomputer or another digital appliance that has processing capabilities(e.g., a portable digital device, such as a mobile phone or smartphone,a laptop computing device, a palmtop computing device, a tabletcomputing device, etc.) As such, the laser-imaging apparatus (i.e., thefundus camera) is operatively coupled to the visual display device ofthe remote operation module 70 so that the digitized fundus (retinal)image of the patient is able to be displayed in detail.

The touchscreen system of the remote operation module 70 comprises adisplay screen that is sensitive to the touch of a finger or a type ofstylus (e.g., a pen stylus). The touchscreen-type visual display deviceincludes: (i) touch sensor in the form of a panel with a responsivesurface, (ii) a hardware-based controller, and (iii) touchscreensoftware executed by the personal computer or other digital appliance.The touch sensor employed by the touchscreen may comprise resistive-typesensor(s), capacitive-type sensor(s), or surface acoustic wavesensor(s). Each of these sensor types has an electrical current passingthrough them, and touching the surface of the screen results in aconsequential voltage change. The voltage change is indicative of thelocation of the touching. The controller is the hardware component thatconverts the voltage changes produced by the touch sensor into signalsthat the personal computer or other digital appliance can receive. Thetouchscreen software instructs the personal computer or other digitalappliance as to what is occurring on the touchscreen and on theinformation delivered from the controller (i.e., what the user istouching and the location of his or her touch) so that the computer ordigital appliance can respond accordingly.

Using the touchscreen visual display device, the physician (i.e., theophthalmologist) can control the laser system with the touch of his orher finger or by using a stylus pen, etc. The visual display device alsoincludes zoom capabilities in order to allow the physician to examinethe details of the digitized fundus image (i.e., retinal image) ifneeded. The display recognizes and interprets the commands of thephysician and communicates those commands to the personal computer thatcontrols the laser generation system. As such, the laser generationsystem is controlled in accordance with the markings made by thephysician on the touchscreen. In other words, the laser generationsystem is configured to carry out the laser treatment in accordance withthe indicated markings on the touchscreen. When a typical procedure isbeing carried out by the physician, the digitized fundus (retinal) imageof the patient is initially recalled on the visual display device.Because the fundus image is digitized, the physician is able toprecisely sketch/draw on any area of the digitized fundus image so thathe or she can indicate the location(s) where the laser pulses should beapplied, and as required, the location(s) where the laser pulses shouldnot be applied. For example, the area or areas where the laser pulsesare to be applied are shaded and/or denoted using a first predeterminedcolor (e.g., green, which signifies a laser is to be applied thereto),while the area or areas where laser pulses are not to be applied areshaded and/or denoted using a second predetermined color (e.g., red,which signifies that a laser is not to be applied in this region orregions). When the laser coagulation system is operating in either thesimulation mode or the actual control mode, any invasion of the areashaded with the second predetermined color (i.e., the red area) willresult in the immediate shutting down of the laser. Also, the area(s)where the laser pulses are to be applied can be separated from the restof the retina by a continuous line of a predetermined color (e.g., a redline) so that there is a clear demarcation on the retinal image. Thearea or areas that are shaded and/or denoted using the secondpredetermined color (i.e., red, indicating that laser pulses are not tobe applied thereto) may represent a part of the retina containing largeretinal vessels, such as the region with the optic nerve head and thefovea, which are sensitive to laser damage and should not be coagulated(e.g., see FIG. 11). The area of the laser coagulation can be contiguousor separated. In general, the distance between each of the laser pulsescan vary (e.g., in the range between 0.1 mm and 4.0 mm or betweenapproximately 0.1 mm and approximately 4.0 mm, inclusive). The laserapplication can be a single pulse as long as it is indicated by a singlecolored dot (e.g., a green dot), or the quantity of the laser spots canrange up to 2500 spots or more. The laser pulse duration can be in therange from one femtosecond to 4.0 seconds, inclusive (or in the rangefrom approximately one femtosecond to approximately 4.0 seconds,inclusive). In other embodiments, the laser pulse duration can exceed4.0 seconds. The energy level of each pulse can range from 0.01femtojoule to one joule, inclusive (or between approximately 0.01femtojoule to approximately one joule, inclusive), and more preferably,between 1 nanojoule to one joule, inclusive (or between approximately 1nanojoule to approximately one joule, inclusive). The spot size of thelaser varies between 0.0001 nanometers (nm) to 2 millimeters (mm),inclusive (or between approximately 0.0001 nm to approximately 2 mm,inclusive).

In addition, in one or more embodiments, the visual display device withtouchscreen capabilities displays the degree of energy (e.g., number oflaser pulses, laser spot size, laser power or laser energy level) andthe number of laser spots applied for a given area. The visual displaydevice also displays all the parameters that are needed or chosen by theoperator in addition to the presentation of the virtual image of thetest laser application. All of this information is recorded and storedin the computer. The remote laser apparatus, which is located at each ofthe local sites 51 a, 51 b, 51 c (see FIG. 5), does not perform thelaser coagulation if the prior test is not performed. Similarly, afterthe correction of any parameter of the laser, or any change in the areaof application of the laser, a new test application is run to ensure thesafety of the procedure.

Also, as a part of the fail safe mechanism of the system, any invasionof the area which is not be subjected to the laser (e.g., the areacolored in red by the physician) results in the immediate shutting downof the laser so that the appropriate inspection can be performed and/orcorrective actions can be taken. As such, the system has completecontrol over the area of laser application.

Referring now to FIGS. 7 and 8, there are shown photographs of theretina, showing the outline of the oval target area (area bound by twooval lines) thereon for application of laser spots (FIG. 7) and thelaser spots (in some embodiments, in the range of approximately 50-3000laser spots) on the retina achieved via virtual treatment (see FIG. 8),as described below. In some embodiments, the physician can draw the ovallines in FIGS. 7 and 8 on the digitized fundus image by utilizing thetouchscreen-type visual display device.

Now, turning to FIGS. 11-14, four (4) images showing exemplary areas oflaser application on a patient are illustrated (e.g., as marked by aphysician using the touchscreen-type visual display device). FIGS. 11-13depict laser application areas in the retina, which have been indicatedon the fundus (retinal) image of a patient, while FIG. 14 depicts alaser application area on the skin surface of a patient. Specifically,in FIG. 11, it can be seen that the annular area 300 (or donut-shapedarea 300) with diagrammatic arrows 301 has been marked for laserapplication (e.g., by using a green color), while the central circulararea 302 has been marked for no laser application thereto (e.g., byusing a red color). As described above, the fovea 304 and the opticnerve head 306, which are located in the central circular area 302, arehighly sensitive to laser damage, and thus, should not be coagulated. Asshown in FIG. 11, the retinal area has a plurality of blood vessels 308disposed throughout; the retinal area is surrounded by the ora serrata310 (i.e., the serrated junction between the retina and the ciliarybody). Next, turning to FIG. 12, two (2) localized areas 312, 314 aremarked for laser application (e.g., by using a green color). As shown inFIG. 12, the localized areas 312, 314 are generally elongated, curvedregions of the retina. Then, with reference to FIG. 13, it can be seenthat a plurality of single localized spots 316 on the retina are markedfor laser application (e.g., by using a green color). As illustrated inFIG. 13, the plurality of localized spots 316 are disposed in closeproximity to the fovea 304 and the optic nerve head 306.

In addition, it is to be understood that the laser coagulation proceduredescribed herein can also be applied to external surface areas on thebody of a patient. For example, in FIG. 14, an area 320 of skin lesion319 on a patient's face 318 is marked for laser application (e.g., byusing a green color). As other examples, the laser coagulation systemdescribed herein could be used to treat the eye surface, the cornea,conjunctiva, eyelid, skin areas, or visible mucosa. As such, any part ofthe eye can be treated with the laser coagulation system describedherein using the same laser-imaging apparatus described above, or amodified camera technology that is particularly suited for surfaceapplications (e.g., a standard camera, such as those that are used inphotography, or a microscope, etc.). Such a camera would be used tocreate images of the eye surface, the cornea, conjunctiva, eyelid, skinareas, or visible mucosa, etc.

Remote Operation and Performance Simulation Module

According to the invention, the remote operation and performancesimulation module 74 allows a physician or technician to perform virtualtreatment, which permits the physician to test the local operationmodule 65 (at a local site) in terms of its generation capability of thelaser spots throughout the surgery area specified by the doctor (seeFIG. 8). The algorithm used in this simulation module is similar to theone used by the local operation module 65.

After the physician (i.e., ophthalmologist) outlines the extent of thedesired treatment area using the touchscreen visual display device ofthe remote operation module 70, he or she initiates the simulationprocess using the remote operation and performance simulation module 74.The treatment area outlined by the physician is filled virtually by themodule 74 with dots (e.g., white dots), thereby covering the area andindicating the extent and the density of the laser spots that shouldoccur on the retina and not on any other area.

Referring now to FIG. 9, in some embodiments, movement of the object(i.e. digitized fundus photograph) is simulated via three randomvariables; X, Y and O. The variables X and Y denote the displacementchanges in the X and Y axes, respectively. The variable O denotes theorientation changes. In a preferred embodiment, the digitized fundusimage is displaced electronically by the remote operation andperformance simulation module 74 using the variables X, Y and O. Thedisplacement of the digitized fundus image represents the movement ofthe retinal image of the patient prior to operating the lasercoagulation system in the actual control mode.

In another embodiment, it is also possible to utilize mechanical meansto displace a hardcopy of the fundus image (e.g., by using amulti-dimensional mechanical actuator that is operatively coupled to thehardcopy of the fundus photograph).

To this end, movement data from test subjects is first collected, fromwhich best-fit statistical distributions (and variances) for the threerandom variables (X, Y, and O) are determined. According to theinvention, a “goodness-of-fit” test can then be performed to test thevalidity of the distribution.

If a theoretical distribution (involving larger p-value in statisticalsense) exists, it will then be employed. Otherwise, an empiricaldistribution will be constructed and employed.

According to the invention, the remote operation and performancesimulation module 74 also allows the physician to test the localoperation module 67 (at a local site) in terms of its generationcapability of the laser spots without involving communications with thelocal control module 62.

Safety and Verification Modules

According to the invention, safety and verification modules 66, 72 existboth at the remote site 58 as well as each local site 52, 54, 56 toensure safe and effective operation of the system 50. In a preferredembodiment of the invention, several constraints are preferably imposedinto the system 50 (both hardware and software). In some embodiments,the constraints include (1) physical constraints, (2) logicalconstraints, and (3) medical constraints.

According to the invention, physical constraints ensure variousparameters or values (e.g., therapy beam power) in the laser system 50are within a permitted range. If any of the values are outside apermitted range, laser firing will be automatically locked andnotification of the unacceptable value(s) is transmitted to thephysician at the remote site, as well as the technician at the localsite.

Logical constraints are employed to ensure that a correct sequence ofoperational tasks is performed. For example, if a physician at a remotesite mistakenly commands 700-1000 laser spots of laser application tothe fundus before simulating the laser application in a simulation mode,the system will not execute the command and transmits a warning messageto the physician. In a preferred embodiment, Unified Modeling Language(UML) is incorporated into the system software and employed to specifythe logical constraints.

The medical constraints involve monitoring of the fundus of the testsubject(s) during the actual laser procedure operation. If it isdetected that the laser energy effect on the fundus is different fromwhat the physician expected or the laser energy is applied beyond thespecified area, laser energy transmission is immediately terminated.Notification of the issue is also transmitted to the physician at theremote site, as well as the physician or technician at the local site.

As indicated above, in some embodiments of the invention, the system 50also includes eye tracking and facial recognition means to ensure safeoperation.

Laser-Imaging System with Image and Voice Recognition Capabilities

Referring now to FIG. 15, there is shown a schematic diagram of alaser-imaging system 50′, according to another embodiment of theinvention. Referring to this figure, it can be seen that, in manyrespects, this embodiment of the laser-imaging system 50′ is similar tothe embodiment of the laser-imaging system 50 described above. Moreover,many elements are common to both such embodiments. For the sake ofbrevity, the elements that the laser-imaging system 50′ has in commonwith the laser-imaging system 50 will only be briefly mentioned, if atall, because these components have already been explained in detailabove. Furthermore, in the interest of clarity, these elements aredenoted using the same reference characters that were used in thepreceding embodiment.

As illustrated in FIG. 15, like the system 50 described above, thesystem 50′ includes a plurality of local control systems disposed atlocal sites 51 a′, 51 b′, 51 c′; each system including a laser-imagingapparatus 55, such as the apparatus 200 shown in FIG. 2. In a preferredembodiment of the invention, each laser-imaging apparatus 55 includes aphotoacoustic system, such as system 80 discussed above.

Preferably, each laser-imaging apparatus 55 of the system 50′ ispreferably in communication with a local control module 62 andcontrol-processing means 59 a. In the illustrated embodiment, thecontrol-processing means 59 a may be embodied as a local personalcomputer or local computing device that is specially programmed to carryout all of the functionality that is described herein in conjunctionwith the local control systems of the laser-imaging system 50′.

In the illustrated embodiment of FIG. 15, each local control system alsoincludes eye tracking means 71 for measuring eye position(s) andmovement. According to the invention, the eye tracking means 71 can bean integral component or feature of the laser-imaging apparatus 55 or aseparate system or device.

Referring again to FIG. 15, it can be seen that each local controlsystem also includes an image recognition sensor 73 for capturing imagesof a subject or patient that may be used to identify and/or verify theidentity of the subject or patient. Advantageously, the positiveidentification and verification of the identity of the subject orpatient receiving treatment prevents surgical mistakes wherein the wrongsubject or patient is treated. In addition, rather than identifyingand/or verifying the identity of the subject or patient, the imagerecognition capabilities of the image recognition sensor 73 may also beused to identify and verify that a particular surgical procedure isbeing performed on the proper body portion of a subject or patient(e.g., to verify that a laser coagulation procedure is being performedon the proper one of the patient's eye or that a surgical procedure isbeing performed on the proper one of the patient's limbs). As will bedescribed hereinafter, in a further embodiment, the image recognitionsensor 73 or imaging device may be provided as part of a dynamic imagingsystem that is operatively coupled to the laser-imaging apparatus 55 ofthe laser treatment system. Also, as will be described hereinafter, theimage recognition sensor 73 or imaging device may comprise a light fieldcamera that is able to simultaneously record a two-dimensional image orthree-dimensional image and metrically calibrated three-dimensionaldepth information of a scene in a single shot. In one or moreembodiments, the image recognition sensor 73 may also comprise amultispectral camera that captures multispectral digital images over awide range of frequencies of the electromagnetic spectrum (e.g., fromvisible light frequencies to infrared radiation frequencies). As such,because the camera is capable of operating using a wide range ofdifferent frequencies, the multispectral camera may be used as both avisible light camera (e.g., operating in the 450-750 nanometer range)and a thermographic camera for capturing images using infrared radiation(e.g., operating with wavelengths up to 14,000 nm (14 μm)).

In one or more embodiments, the image recognition sensor 73 of eachlocal control system may be operatively connected to the local computingdevice, which forms the control-processing means 59 a of the localcontrol system. The local computing device may be specially programmedwith image/pattern recognition software loaded thereon, and executedthereby for performing all of the functionality necessary to identifyand verify a particular subject or patient, or to identify and verify abody portion of the particular subject or patient that is to be treated.Initially, the local computing device may be specially programmed tocapture and store a first reference digital image of a body portion ofthe subject or patient so that the first reference digital image may becompared to a second digital image of the same body portion captured ata later time (i.e., just prior to the performance of the surgicalprocedure). Then, prior to the performance of the surgical procedure(e.g., a laser coagulation performed on the eye), the second digitalimage of the same body portion of the subject or patient is captured bythe image sensor 73 (i.e., the multispectral camera) and the localcomputing device compares the second digital image of the body portionto the first reference digital image and determines if the seconddigital image of the body portion matches or substantially matches thefirst reference digital image. When the local computing devicedetermines that the second digital image of the body portion of thesubject or patient matches or substantially matches the first referencedigital image, the local computing device is specially programmed togenerate a matched image confirmation notification that is sent to theremote computing device at the remote site in order to inform theattending physician that the proper patient or body portion of thepatient has been identified and verified. The matched image confirmationnotification may also be delivered to the technician at the local sitevia the local computing device. Then, after the other safety checks ofthe system 50′ have been performed, the surgical procedure is capable ofbeing performed on the patient. Conversely, when the local computingdevice determines that the second digital image of the body portion ofthe subject or patient does not match or substantially match the firstreference digital image, the local computing device is speciallyprogrammed to generate a non-matching image notification that is sent tothe remote computing device at the remote site in order to inform theattending physician that the patient or body portion of the patient hasnot been properly identified and verified. The non-matching imagenotification may also be delivered to the technician at the local sitevia the local computing device. When the non-matching image notificationis sent to the attending physician, the local computing device alsodisables the surgical equipment at the local site in order to preventthe procedure from being performed on the incorrect patient or theincorrect body portion of the patient (e.g., in a laser coagulationprocedure, laser firing will be automatically locked out by the localcomputing device).

When the multispectral camera comprising the image recognition sensor 73is used as a thermographic camera in the mid-infrared or infraredradiation wavelength range, the camera may be used to capture images ofcapillaries in the skin of the patient or subject. As such, the cameramay sense the heat generated by the blood in these capillaries toidentify the subject or patient, or a particular body portion of thesubject or patient. A multispectral camera that is capable of bothvisible light photography and infrared pattern recognition takes intoaccount psychological functions which are not achievable with a camerahaving only visible light capabilities (e.g., two or three differentphysiological functions of the patient may be taken into account with amultispectral camera). For example, prior to capturing the infraredimage of a face of the subject or patient, the subject or patient may beinstructed to frown so that wrinkles are made prominent in the face ofthe subject or patient. When the subject or patient frowns, skincapillaries become collapsed and folded, thereby reducing the blood flowthrough the collapsed and folded capillaries and the heat that isdetected by the multispectral camera. As such, the local computingdevice may be specially programmed in the manner described above toverify the identity of the subject or patient using the image patterncreated by this particular facial expression (i.e., frowning).Detectable wrinkles may also be created by instructing the patient tolift up his or her brow, by instructing the patient to laugh, or byinstructing the patient to execute any other facial expression thatcreates wrinkles. As another example, the multispectral camera may beused to capture a body portion of the subject or patient (e.g., aportion of the patient's skin) that has previously undergone a surgicalprocedure whereby the structure of the body portion has been altered(e.g., the capillaries in a portion of the patient's skin that has beenpreviously operated on will have a very unique pattern that is specificto the particular patient being verified). The scars created by thesurgery create a very different, unique marker on the patient (e.g.,scar tissue created by surgery does not have many capillaries). Becausethis specific pattern of skin capillaries that were surgically alteredis extremely hard to artificially replicate, this means of verifying apatient's identity is very reliable and imposters are highly unlikely.

Now, turning to FIG. 16, exemplary image capture areas on a face 402 ofa patient 400 are shown. These are merely illustrative examples of someareas that may be imaged by the image recognition sensor 73 of the localcontrol system. The first image capture area 404 in FIG. 16 is disposedaround the left eye of the patient 400, and includes the left eye plusthe area surrounding the left eye (e.g., so that facial features, suchas wrinkles around the eye, may be used for patient identification andverification purposes in addition to the eye itself). The second imagecapture area 406 in FIG. 16 is disposed around the left corner of themouth of the patient 400, and includes a portion of the mouth and thearea proximate to the mouth (e.g., so that facial features, such aswrinkles around the mouth, may be used for patient identification andverification purposes). The third image capture area 408 in FIG. 16 isdisposed around the entire face 402 of the patient 400 (e.g., so that aplurality of different facial features, such as wrinkles disposedthroughout the face, may be used for patient identification andverification purposes). When the image of the face 402 of the patient400 is captured by the image recognition sensor 73, the wrinkles in theface 402 remain dark. It is very difficult to circumvent the imagerecognition system by replicating wrinkles. The fourth image capturearea 410 in FIG. 16 is disposed around the right eye of the patient 400(e.g., so that the various structures of the right eye, such as the irisand retina, may be used for patient identification and verificationpurposes).

In addition to the face 402 of the patient 400, it is to be understoodthat other body surfaces may be imaged by the image recognition sensor73 of the local control system as well. For example, any portion of thepatient's body where the physician would like to apply a laser lesionunder observation may be imaged by the image recognition sensor 73.Laser applications may vary from a nearly invisible laser lesionresponse from the tissue to an obvious burn of a given size or fortissue ablation and evaporation using any suitable laser, such as a gaslaser, a solid-state laser, an ultraviolet laser, an infrared laser(i.e., a mid-infrared laser up to a CO₂ laser for surface tumor removalor for rejuvenation for cosmetic surgery), an excimer laser (i.e., forcorneal surgery) with wavelengths of approximately 192 nm, and a CO₂laser with wavelengths of approximately 10,000 nm (10 micron). Othersuitable excimer lasers with various wavelengths may also be used (i.e.,excimer lasers with other ultraviolet wavelengths). In addition,microwave or radiofrequency radiation may be applied to the patient froman appropriate unit to achieve the desired therapeutic result. The beamdiameter may be varied or be focused to a diameter of one micron to fivemillimeters or more. The duration of each pulse can vary from onefemtosecond to a continuous laser application depending on the need ofthe operator to achieve a particular result.

In a further embodiment, the laser-imaging system 50′ may comprise animage recognition sensor 73 in the form of a holoscopicthree-dimensional (3D) camera at the local site (i.e., to capture 3Dimages of the patient), and the graphical user interface of the commandcomputer 59 b at the remote site may be in the form of a multiview,three-dimensional (3D) display (i.e., so the physician can view the 3Dimages of the patient for identification purposes). In this furtherembodiment, changes in the physiological condition of the patient areinduced (e.g., such as by having the patient smile, frown, etc.) whilethe changes are recorded by the 3D camera before, during, and after amedical procedure. Then, the results are displayed to the physician atthe remote site using the multiview 3D display. Advantageously, themultiview 3D display permits a three-dimensional analysis of thephysiological changes of the patient by the physician before and after aphysiological change is induced. Also, the physician is able to analyzethe trends of the physiological changes of the patient, such asaugmentation or diminution of the physiological change. Also, thephysical parameters associated with the physiological changes are ableto be analyzed using a computer at the remote site (i.e., computer 59b), and compared with the initial existing data.

In this further embodiment, changes in the physiological condition maybe induced in the patient after the patient responds to a question posedto him or her. In this case, the surface changes in the patent's facewhile answering the question may be recorded by the image recognitionsystem and analyzed for the patient's recognition by comparing therecorded results to a previous session. Also, the patent's speech may berecorded while he or she is answering the question so that the speechpatterns of the patient may be analyzed using the voice recognitionsensor 75 described hereinafter.

Also, in this further embodiment, the multiview, three-dimensional (3D)display of the image recognition system may comprise a digitalholographic display so as to allow the physician to perform athree-dimensional analysis of the changes in the physiological conditionof the patient which are induced (e.g., such as by having the patientsmile, frown, etc.). In this further embodiment, the digital holographicdisplay may permit 3D display while maintaining the phase and amplitudedata to reconstruct the wavefront through the use of a stereoscopic headmount display or console to reduce the sensory conflict of the eyeduring the convergence and accommodation and aid in a virtual realitydisplay of the head, face, extremities, retina, vessels etc., for threedimensional analysis of the physiological changes before and after thephysiological change is induced and the trends of the physiologicalchanges. Also, the 3D display may be in the form of virtual realityglasses that are worn by the treating physician at the remote site.

Moreover, in this further embodiment, the digital holographic display ofthe image recognition system may comprise one or more thin holographicoptical elements or one or more ultrathin optical elements for producinghigh-resolution 3D images, and may further comprise a multiviewautostereoscopic 3D display to eliminate the need for the physician towear special eye wear while performing the three-dimensional analysis ofthe induced changes in the physiological condition of the patient.

Furthermore, in this further embodiment, the multiview,three-dimensional (3D) display of the image recognition system maycomprise a volumetric 3D display so as to present a 3D image on whicheach voxel is appropriately positioned, and reflects light to form areal image for the viewer (i.e., the physician) providing physiologicaland perceptual hints and spatial depth, and volume with high resolutionof a structure while performing the three-dimensional analysis of theinduced changes in the physiological condition of the patient.

Additionally, in this further embodiment, the holoscopicthree-dimensional (3D) camera of the image recognition system may beconfigured to record tomographic images from a surface structure of thepatient, such as the skin, as deep as the light can penetrate the tissuefor three-dimensional analysis of the induced physiological changes andtheir trends. Also, the induced physiological changes of the patient maybe additionally recorded during and after imaging using a near-infraredlaser with Optical Coherence Tomography (OCT) or Near-Infrared opticalTomography (NIROT). When OCT is used for recording the changes,wide-angle Optical Coherence Tomography (OCT) may be utilized so thatchanges, such as the elevation or depression of a structure (e.g., thecheek, nose, tumor of the skin, etc.), is capable of being used during atwo or three-dimensional analysis of the physiological changes and theirtrends. In addition, the induced physiological changes of the patientalso may be recorded using a Frequency Modulated Continuous Wave (FMCW)system with ranging detection so as to obtain before, during, and afterthe physiological change is induced, 3D images of a surface whichdiffuses light with precision, i.e., under 10 micron speckle noise,while permitting the analysis of a variety of reflecting materials andtissue.

In another embodiment, the dynamic identity imaging involves analysis ofa person's physical or psychological changes by presenting to a subject,a range of images and/or sounds or videos of incidences of relaxation orpleasure to extreme sadness or pain, while recording the person's bodyor the face with a hyperspectral camera, etc. to observe his or herresponse to the images/sound that can be analyzed in a dynamic formatfor physical or physiological, medical, psychology, psychiatric, orinvestigational, medical diagnosis, or security field, etc. orevaluation of a symptom of a disease before and after therapy, or toobserve children in the playground, etc.

In yet a further embodiment, the image recognition sensor 73 of thelaser-imaging system 50′ may comprise two spaced-apart cameras thatcapture two standard images separated by 60 to 70 millimeters (mm) fromeach other (e.g., like two eyes of human, but as small as the camera ona smartphone). The two small cameras may be mounted over thephotographic and imaging system of the laser-imaging system 50′. Twoseparate, but simultaneous images are taken with a white flash lightgenerating a full spectrum of color (ultraviolet to infrared orinfrared, etc.) using a diode laser. The patient can perform a similaraction as that described above, such as smiling or frowning, which isthen recorded by the two small cameras. The image is transmittedseparately, but simultaneously via the Internet to the computer at theremote site (i.e., computer 59 b), which may be located at thephysician's or observer's office. This information is transmitted to areceiver and a small processor worn by the physician or observer, andconverted to two different light pulses via a fiber optic to a smallprismatic mirror located in front of each pupil of physician orobserver, and projected into the eyes of the doctor's or observer'sretina so that the right eye sees the images from the right camera andthe left eye sees the images from the left camera. Both cameras may beworking as a video system so that the doctor or observer sees a 3D videoof the patient's face from the time that the patient sits in front ofthe cameras up to the time that the treatment laser of the laser-imagingsystem 50′ is applied to the patient's face or to another part of thepatient's body. Using the camera(s), it is also possible to create a 3Dimage of the retina or an object, such as face, located close to thecamera(s). Also, in this further embodiment, an electronicallycontrolled rapidly moving prism (e.g., oscillating at greater than 60Hertz) or an electronically controlled rotatable lens (e.g., rotatingback and forth by 10 to 30 degrees) in front of a single camera may beprovided so as to create two different, but slightly angulated imagesfrom an outside object (e.g., face etc.) or from inside the eye of apatient. These separate images again are sent via the Internet to botheyes of the physician or an observer separately, but with the samefrequency (greater than 60 Hertz). These images are capable of creatingthe sensation of a 3D display for the physician or observer who sees twodifferent images with both eyes, but because of the frequency of theimage presentation, the images are fused in the brain and seen as onethree-dimensional image. In addition, the computer at the remote site(i.e., computer 59 b) may also be configured to separately analyze eachframe of the video so as to analyze changes that have occurred in thephysiological condition of the patient by virtue of the induced changes(e.g., smiling, frowning, etc.) so that the patient's identity may beverified before any treatment is initiated.

In still a further embodiment, the image recognition system of thelaser-imaging system 50′ may comprise a three-dimensional (3D)multi-color meta-holography device for recording, analyzing andtransmitting the data obtained during patient observation as a result ofchanges during and after a physiological action of the patent so as todistinguish newly computer-generated data from the previously storeddata. This embodiment utilizes metasurface, meta-hologram imaging and 3Dmeta-holographic imaging that has been made possible by advancement innanotechnology. In this embodiment, increased data capacity is achievedby the incorporation of single or different-sized plasmonic pixels intoa metasurface, so as to create 3D multicolor meta-holography withreduced overlap between different colors. In this embodiment, the imagerecognition sensor 73 of the laser-imaging system 50′ is in the form ofa 3D multicolor meta-holographic imaging device and, in case of adiscrepancy, the image recognition system is configured to shut down thelaser treatment system automatically while the doctor or observerlocated in at the remote site simultaneously views the 3D multicolormeta-holographic images produced by the image recognition system.

In yet a further embodiment, the photoacoustic system 80 described abovemay be used for ultrasonic three-dimensional (3D) imaging of bodystructures beneath the skin of the patient, such as bone structures inthe body of the patient. For example, the transducer array of thephotoacoustic system 80 may be used for obtaining an ultrasonic 3D imageof the bone structure of the nose of the patient while the head ofpatient is supported on a head support structure disposed in front ofthe photoacoustic system 80. Because the nose bone structure is uniqueto each patient, the imaged bone structure of the nose also may be foridentification purposes in the manner described above for skinstructures.

Turning again to FIG. 15, it can be seen that each local control systemmay further include a voice recognition sensor 75 for capturing thespeech waveforms generated by the subject or patient so that the speechof the subject or patient may additionally be used to identify and/orverify the identity of the subject or patient. In one or moreembodiments, the voice recognition sensor 75 may be used in conjunctionwith the image recognition sensor 73 described above to further verifythat a surgical procedure is being performed on the correct subject orpatient. In one or more embodiments, the voice recognition sensor 75 maycomprise a microphone that captures the speech of the subject or patientover the entire speech frequency range of a human being (e.g., for afrequency range from 50 Hz to 5,000 Hz to encompass the typicalfrequency range for both males and females). As such, the syntax andsound pulses generated by the subject or patient are capable of beingused by the local control system for verification and identification ofthe subject or patient prior to the surgical procedure being performedon him or her. In one or more embodiments, the voice recognition sensor75 may be used as a second means of patient identity confirmation inorder to confirm the identity of the patient that was previouslyverified by the image recognition sensor 73. In other words, the imagerecognition sensor 73 may comprises a first stage of patient identityconfirmation, and the voice recognition sensor 75 may comprise a secondstage of patient identity confirmation.

Similar to the image recognition sensor 73 described above, the voicerecognition sensor 75 of each illustrative local control system may beoperatively connected to the local computing device, which forms thecontrol-processing means 59 a of the local control system. The localcomputing device may be specially programmed with voice recognitionsoftware loaded thereon, and executed thereby for performing all of thefunctionality necessary to identify and verify a particular subject orpatient that is to be treated. Initially, the local computing device maybe specially programmed to capture and store a first reference speechwaveform of the subject or patient so that the first reference speechwaveform may be compared to a second speech waveform of the same patientor subject captured at a later time (i.e., just prior to the performanceof the surgical procedure). That is, the patient or subject may be askedto say a particular word or plurality words that are captured by thevoice recognition sensor 75 so that it can be used as the firstreference speech waveform. Then, prior to the performance of thesurgical procedure (e.g., a laser coagulation performed on the eye), thesecond speech waveform of the subject or patient is captured by thevoice sensor 75 (i.e., the microphone records the same word or pluralitywords repeated by the subject or patient) and the local computing devicecompares the second speech waveform of the patient or subject to thefirst reference speech waveform and determines if the second speechwaveform of the subject or patient matches or substantially matches thefirst reference speech waveform (i.e., by comparing the frequencycontent of the first and second speech waveforms). When the localcomputing device determines that the second speech waveform of thesubject or patient matches or substantially matches the first referencespeech waveform, the local computing device is specially programmed togenerate a matched speech confirmation notification that is sent to theremote computing device at the remote site in order to inform theattending physician that the proper patient has been identified andverified. The matched speech confirmation notification may also bedelivered to the technician at the local site via the local computingdevice. Then, after the other safety checks of the system 50′ have beenperformed, the surgical procedure is capable of being performed on thepatient. Conversely, when the local computing device determines that thesecond speech waveform of the subject or patient does not match orsubstantially match the first reference speech waveform, the localcomputing device is specially programmed to generate a non-matchingspeech notification that is sent to the remote computing device at theremote site in order to inform the attending physician that the patienthas not been properly identified and verified. The non-matching speechnotification may also be delivered to the technician at the local sitevia the local computing device. When the non-matching speechnotification is sent to the attending physician, the local computingdevice also disables the surgical equipment at the local site in orderto prevent the procedure from being performed on the incorrect patient(e.g., in a laser coagulation procedure, laser firing will beautomatically locked out by the local computing device).

Similar to that described above for the system 50 of FIG. 5, thelaser-imaging system 50′ of FIG. 15 also includes a central controlsystem 58 at a remote site having a command computer 59 b that isoperatively connected to a remote control module 64. Also disposed atthe remote site 58 during a laser procedure is a system operator (e.g.,retinal surgeon). In FIG. 15, it can be seen that the central controlsystem 58 at the remote site, which includes the command computer 59 b,is operatively connected to the plurality of local control systemsdisposed at local sites 51 a′, 51 b′, 51 c′ via a computer network thatuses the Internet 60.

In one or more further embodiments, laser-imaging system 50′ may furtherinclude a dynamic imaging system comprising an imaging device configuredto capture images of a body portion of a person over a predeterminedduration of time so that a displacement of the body portion of theperson is capable of being tracked during the predetermined duration oftime; and a data processing device (which may be the local computingdevice 59 a) operatively coupled to the imaging device, the dataprocessing device being specially programmed to determine thedisplacement of the body portion of the person over the predeterminedduration of time using the captured images, and to compare thedisplacement of the body portion of the person over the predeterminedduration of time to a reference displacement of the body portion of theperson acquired prior to the displacement so that dynamic changes in thebody portion of the person are capable of being assessed (e.g., by thelocal computing device 59 a and the software algorithm executed thereby)for the purpose of identifying the person or evaluating physiologicalchanges in the body portion. In particular, the local computing device59 a may execute a subtraction algorithm to demonstrate and project thespecific areas of the body or face and ignore the rest of the data toclearly present weighted pixelated data for the future comparison of thesame body or face and the trends of the area(s) of displacement. Inthese one or more further embodiments, at least two points ofdisplacement may be used for the comparison of the subsequentdisplacement of the body portion of the person to the referencedisplacement and its recorded displacement trend(s).

In one or more embodiments, the subtraction algorithm or analysisperformed by the local computing device 59 a may subtract a secondsubsequent image from a first image in order to isolate the portion ofthe second image that changed as compared to the first image. In otherwords, the subtraction algorithm or analysis performed by the localcomputing device 59 a removes or subtracts out the portion of the firstand second images that is unchanged between the two images so as toisolate only the changed image portion. In one or more embodiments, thedynamic imaging procedure is a two-step process. During the first step,a first image of the patient is taken (e.g., the face of the patient)without requiring the patient to perform any particular task (e.g.,smile or frown). During the second step, a second image of the patientis taken (e.g., the face of the patient) while requiring the patient toperform a particular task (e.g., smile or frown). Then, the first andsecond images are compared by the local computing device 59 a using thesubtraction algorithm or analysis. For example, if the patient is askedto smile or frown during the dynamic imaging procedure, only the mouthportion of the facial image of the patient is changed, thus thesubtraction algorithm or analysis performed by the local computingdevice 59 a eliminates the remainder of the facial image that isunchanged in order to enhance the processing speed of the image analysisprocess.

Because the second step of the two-step process dynamic imagingprocedure described above requires the patient to perform an action whenasked to do so by the clinician, the two-step dynamic imaging procedureresults in the patient giving implicit consent to the dynamic imagingprocedure (e.g., for identification purposes) because the person mustvoluntarily execute the task that is requested of him or her prior tothe second image being taken. As such, consent issues that arise withconventional patient identification methods are averted by the dynamicimaging system described herein.

In one or more embodiments, the local computing device 59 a is furtherprogrammed to determine a confidence level of the patient identificationprocedure performed by the dynamic imaging system herein. For example,the local computing device 59 a may determine with 98% certainty that asubsequent image is of the same patient as that depicted in an initialreference image. The local computing device 59 a may further beprogrammed to determine a percentage change between a first image and asecond image (e.g., a percentage change in the pixels of a first tumorimage and a second tumor image). The local computing device 59 a mayadditionally be programmed to reconstruct an incomplete image of apatient based upon a previous image of the same patient (e.g.,reconstruct an incomplete facial image).

In one or more further embodiments, the image recognition sensor 73 orimaging device of each local control system may be operatively connectedto the local computing device, which forms the control-processing means59 a of the local control system. The local computing device may bespecially programmed with image/pattern recognition software loadedthereon, and executed thereby for performing all of the functionalitynecessary to identify and verify a particular subject or patient, or toidentify and verify a body portion of the particular subject or patientthat is to be treated, the local computing device may be connected viathe internet to the remote location where the operator or a doctor canuse a helmet or gloves fitted with sensors or use the personal computer59 b with a mouse and software capable of creating an artificialenvironment where the patient or the subject is located (e.g. in aremote office as an augmented reality), so that the doctor or anobserver can obtain all the data from the patient's body, face, orretina, etc. prior to or during or after dynamically inducedphysiological changes in the patient or subject, so that operator or thedoctor can move the content of the three-dimensionally created imagefrom one to another direction or zoom in or out or using presentlyavailable technology, such as Google® Glass or a heads-up display with asmall screen in front of the eyes, etc. or as a wrap-around displayscreen as virtual reality to simulate the dynamic observation of thechanges, the trend and subtraction of the before and after images by thecomputer algorithm to look objectively at the final product for thefirst time or compare it in the future session to another dynamic imagesimilarly obtained from the patient or the subject to observe thedifferences or trends of the final subtracted data in the patient's orsubject's image in the same environment where the patient is located,and thus creating an augmented reality or an artificial environment asvirtual reality away from the patient or the subject and storing theinformation and data obtained after treatment of the patient in thecloud or another computer after encrypting the data for futurereference.

In these one or more further embodiments, the image recognition sensor73 or imaging device of each local control system may be operativelyconnected to the local computing device, which forms thecontrol-processing means 59 a of the local control system. The localcomputing device may be specially programmed with image/patternrecognition software loaded thereon, and executed thereby for performingall of the functionality necessary to identify and verify a particularsubject or patient, or to identify and verify a body portion of theparticular subject or patient that is to be treated. Initially, thelocal computing device may be specially programmed to capture and storereference dynamic information regarding a body portion of the subject orpatient so that the reference dynamic information may be compared todynamic information pertaining to the same body portion captured at alater time (i.e., just prior to the performance of the surgicalprocedure). Then, prior to the performance of the surgical procedure(e.g., a laser coagulation performed on the eye, or body surface, etc.),dynamic information pertaining to the same body portion of the subjector patient is captured by the image sensor 73 (i.e., the light fieldcamera) and the local computing device compares the subsequent dynamicinformation pertaining to the body portion to the reference dynamicinformation and determines if the subsequent dynamic information of thebody portion matches or substantially matches the reference dynamicinformation. In one or more embodiments, this information may relate tothe position or size of any part of the body, extremities, or organ(e.g., as seen on an X-ray film, MRI, CT-scan, or gait changes bywalking or particular habit of head position, hand, facial changesduring speech, or observation of changes due to emotional changes,etc.). When the local computing device determines that the subsequentdynamic information pertaining to the body portion of the subject orpatient matches or substantially matches the reference dynamicinformation, the local computing device is specially programmed togenerate a matched identity confirmation notification or the trends ofchanges, etc. in a two-dimensional and/or three-dimensional manner thatis sent to the remote computing device at the remote site in order toinform the attending physician or the receiving authority that theproper patient or body portion of the patient has been identified andverified or his or her response to stimuli. The matched identityconfirmation notification may also be delivered to the technician at thelocal site via the local computing device. Then, after the other safetychecks of the system 50′ have been performed, the surgical procedure iscapable of being performed on the patient. Conversely, when the localcomputing device determines that the subsequent dynamic informationregarding the body portion of the subject or patient does not match orsubstantially match the reference dynamic information, the localcomputing device is specially programmed to generate a non-matchingidentity notification that is sent to the remote computing device at theremote site in order to inform the attending physician that the patientor body portion of the patient has not been properly identified andverified. The non-matching identity notification may also be deliveredto the technician at the local site via the local computing device. Whenthe non-matching identity notification is sent to the attendingphysician, the local computing device also disables the surgicalequipment at the local site in order to prevent the procedure from beingperformed on the incorrect patient or the incorrect body portion of thepatient (e.g., in a laser coagulation procedure, laser firing will beautomatically locked out by the local computing device).

In these one or more further embodiments, turning again to FIG. 15, itcan be seen that each local control system may further include a voicerecognition sensor 75 for capturing the speech sound waves generated bythe subject or patient so that the speech of the subject or patient mayadditionally be used to identify and/or verify the identity of thesubject or patient, and his or her gender according to the frequenciesof the sound waves and using, similar to the imaging system, asubtraction analysis of the sound initially and in the subsequentsessions. In one or more embodiments, the voice recognition sensor 75may be used in conjunction with the image recognition sensor 73 orimaging device described above to further verify that a surgicalprocedure is being performed on the correct subject or patient. Thevoice recognition sensor may be simultaneously overlaid on the dynamicimage recognition changes for facial or body recognition. Particularly,in one or more embodiments, the sound waves generated from the dataacquired by the voice recognition sensor 75 may be superimposed on thedisplacement curve generated from the data acquired by the imagingdevice (e.g., the light field camera) so that both audial and visualattributes of the person may be taken into account for identificationpurposes, thereby making the identification of the person far moreaccurate. For example, as a person recites a series of vowels (i.e.,AEIOU), the displacement of the lips of the person is recorded by theimaging device (e.g., the light field camera), while the voicerecognition sensor 75 simultaneously records the voice of the person(e.g., the amplitude and/or frequency of the sound wave generated by theperson). In one or more embodiments, the dynamic imaging system is usedalone or simultaneously with voice overlay, thereby creating 2D or 3Dimages using multispectral light, which includes IR and mid IR, tomeasure time dependent changes for creation of a dynamic event made ofvoice and images or changes thereof. In one or more embodiments, asubtraction algorithm of the system is used to produce a clearsubtracted wave complex from a person examined a first and second time,and to project the subtracted wave complex on the subtracted value ofthe person's image so as to evaluate a match or change, and present thechanged values or their difference as compared to the sound wavesreceived the first time by the voice recognition sensor 75. In one ormore embodiments, the voice recognition sensor 75 may comprise amicrophone that captures the speech of the subject or patient over theentire speech frequency range of a human being (e.g., for a frequencyrange from 50 Hz to 5,000 Hz to encompass the typical frequency rangefor both males and females). As such, the syntax and sound frequenciesgenerated by the subject or patient are capable of being used by thelocal control system for verification and identification of the subjector patient prior to the surgical procedure being performed on him orher. In one or more embodiments, the voice recognition sensor 75 may beused as a second means of patient identity confirmation in order toconfirm the identity of the patient that was previously verified by theimage recognition sensor 73 or imaging device. In other words, the imagerecognition sensor 73 or imaging device may comprise a first stage ofpatient identity confirmation, and the voice recognition sensor 75 maycomprise a second stage of patient identity confirmation.

Similar to the image recognition sensor 73 or imaging device describedabove, the voice recognition sensor 75 of each illustrative localcontrol system may be operatively connected to the local computingdevice, which forms the control-processing means 59 a of the localcontrol system. The local computing device may be specially programmedwith voice recognition software loaded thereon, and executed thereby forperforming all of the functionality necessary to identify and verify aparticular subject or patient that is to be treated. Initially, thelocal computing device may be specially programmed to capture and storea first reference speech waveform of the subject or patient so that thefirst reference speech waveform may be compared to a second speechwaveform of the same patient or subject captured at a later time (i.e.,just prior to the performance of the surgical procedure). That is, thepatient or subject may be asked to say a particular word, a plurality ofwords, or a series of vowels (i.e., AEIOU) that are captured by thevoice recognition sensor 75 so that it can be used as the firstreference speech waveform. Then, prior to the performance of thesurgical procedure (e.g., a laser coagulation performed on the eye orthe body), the second speech waveform of the subject or patient iscaptured by the voice sensor 75 (i.e., the microphone records the sameword, plurality of words, or series of vowels repeated by the subject orpatient) and the local computing device compares the second speechwaveform of the patient or subject to the first reference speechwaveform and determines if the second speech waveform of the subject orpatient matches or substantially matches the first reference speechwaveform (i.e., by comparing the frequency content of the first andsecond speech sound waves). When the local computing device determinesthat the second speech waveform of the subject or patient matches orsubstantially matches the first reference speech waveform, the localcomputing device is specially programmed to generate a matched speechconfirmation notification that is sent to the remote computing device atthe remote site in order to inform the attending physician that theproper patient has been identified and verified. The matched speechconfirmation notification or its discrepancies may also be delivered tothe technician at the local site via the local computing device. Then,after the other safety checks of the system 50′ have been performed, thesurgical procedure is capable of being performed on the patient.Conversely, when the local computing device determines that the secondspeech waveform of the subject or patient does not match orsubstantially match the first reference speech waveform, the localcomputing device is specially programmed to generate a non-matchingspeech notification that is sent to the remote computing device at theremote site in order to inform the attending physician that the patienthas not been properly identified and verified. The non-matching speechnotification may also be delivered to the technician at the local sitevia the local computing device. When the non-matching speechnotification is sent to the attending physician, the local computingdevice also disables the surgical equipment at the local site in orderto prevent the procedure from being performed on the incorrect patient(e.g., in a laser coagulation procedure, laser firing will beautomatically locked out by the local computing device). In one or moreembodiments, the voice changes representing various emotional stages ofthe patients may be recorded for diagnosis or excessive stimuli, such asemotion or pain causing those changes, so that the power of a laser isaccordingly adjusted, etc.

In one embodiment, the local computing device is further programmed toask the patient a question so that the patient is able to respond to thequestion posed to him or her using natural language. That way, thesystem is not only able to analyze the physical surface attributes ofthe patient, but also analyze the sound of the patient's voice (i.e., byvoice recognition recording), and communicate simultaneously withaccessible existing data in a computer database to verify the identityof the patient.

In one embodiment, the time and the information obtained from thepatient is recorded and stored for the future recall. In one embodiment,for a patient whose history does not yet exist in the computer database,the information will be recorded so as to be recognized by the systemthe next time the patient comes for reexamination.

In one embodiment, the system can access other computers searching forthe existence of similar cases and photos of a surface lesion (e.g.,X-ray, CT-scan, MRI, PET-scan, etc. of a lesion). The system may also beused as a telesystem accessing existing data in the published literatureto assist the doctor with the diagnosis and further therapyrecommendation for the patient.

In one embodiment, the computer system functions as an informationalunit augmenting the knowledge of the doctor and assists in presentinghim or her with similar recorded cases to assist in a bettertelemedicine diagnosis and management.

In one embodiment, the system may augment recognition of the patient byusing additional information from a fingerprint, etc. For example,information regarding the tip of the patient's finger may be recorded(e.g., the blood circulation in the finger as well as images of theridges and minutiae, etc. of the fingerprint) in a two-dimensionaland/or three-dimensional manner so as to be used for identifying thepatient in conjunction with the dynamic facial recognition of the personin a two-dimensional and/or three-dimensional manner. As such, thesystem advantageously provides augmented dynamic identification of thepatient.

In one embodiment, the system works as an augmented intelligenceassistant so as to assist a person in to making a proper decision (e.g.,proper treatment of a cancer patient).

In one embodiment, the information obtained by the system is encryptedand transmitted so as to make it virtually impossible to be hacked by athird party.

In one embodiment, the system can differentiate between a human personand robot by its multispectral camera analysis recording the body'stemperature versus the relatively cold body of a robot and thereflective properties of its body surface, etc.

In one embodiment, the system can differentiate between a person havinga dark skin pigmentation and another person with lighter skinpigmentation by its multispectral camera analysis using the infrared(IR) spectrum of the camera to obtain images during the dynamic facialrecognition that can see the skin capillaries under the skin because ofselective penetration of IR wavelength of the light through the skinpigment and observe changes (e.g., collapse of capillaries at specificsites around the lips and in the eyebrows or forehead that fold and thecollapse the capillaries inside the folds) during the dynamic facialrecognition (e.g., with smiling or frowning), creating distinctlandmarks that can be recognized by the subtraction of images using thesoftware algorithm or observing additional landmarks, such as the teethduring smiling.

As mentioned above, in the illustrative embodiment, the imagerecognition sensor 73 or imaging device at the remote laser deliverysite may be in the form of a digital light field photography (DLFP)camera with microlenses that capture the information about the directionof the incoming light rays and a photosensor array that is disposedbehind the microlenses. A specially programmed data processing device(e.g., the local computing device 59 a) is used to process theinformation obtained from the light field camera by converting it intotwo or three dimensional images, or the data is transmitted to a virtualreality system to be observed and recorded by the examiner or a doctor.

In one or more embodiments, the light field digital camera or digitallight field photography (DIFLFP) camera comprises one or more fixedoptical element(s) as the objective lens providing a fixed field of viewfor the camera. A series of microlenses are located at the focal pointof the objective lens in a flat plane perpendicular to the axial rays ofthe objective lens. These microlenses separate the incoming rays oflight entering the camera into individual small bundles. The individualsmall bundles of light are refracted on a series of light sensitivesensors that measure in hundreds of megapixels, which are located behindthe plane of the microlenses, thereby converting the light energy intoelectrical signals. The electronically generated signals conveyinformation regarding the direction of each light ray, view, and theintensity of each light ray to a processor or a computer. Each microlenshas some overlapping view and perspective from the next one which can beretraced by an algorithm. Appropriate software and algorithmsreconstruct computer-generated 2-3 D images from the objects not only infocus, but also those located in front or in the back of the object from0.1 mm from the lens surface to infinity, which is being photographed byretracing the rays via the software and algorithm that modifies, ormagnifies the image, as desired, while eliminating electronically theimage aberrations, reflections, etc. As such, by using a digital camera,one can manipulate the image data by using an appropriate algorithm soas to create new in focus image(s) mathematically.

In one or more embodiments, the light sensitive sensors behind thelenslets of the camera record the incoming light and forward it aselectrical signals to the camera's processor and act as an on/off switchfor the camera's processor measuring the intensity of the light throughits neuronal network and its algorithm to record changes in lightintensity, while recording any motion or dynamic displacement of anobject or part of an object in front of the camera in a nanosecond to amicrosecond of time. The processor of the camera with its neuronalnetwork algorithm processes the images as the retina and brain in ahuman being functions by finding the pattern in the data and its dynamicchanges of the image and its trend over a very short period of time(e.g., nanosecond). The information is stored in the memory system ofthe camera's processor, as known in the art, as memory resistor(memristor) relating to electric charge and magnetic flux linkage, whichcan be retrieved immediately or later, and further analyzed by the knownmathematical algorithms of the camera and can be used for many differentapplications, in addition to applications for 2-3 D dynamic imagerecognition as near or remote subjects recognition, incorporated in theremote laser delivery system described above.

In one or more embodiments, the light field camera may have either atunable lens or a fluidic lens. If a tunable lens is utilized, thetunable lens may be in the form of a shape-changing polymer lens (e.g.,an Optotune® lens), a liquid crystal lens, an electrically tunable lens(e.g., using electrowetting, such as a Varioptic® lens). Alternatively,a fluidic lens may be used in the light field camera.

In one or more embodiments, the light field camera may have a simpleflexible transparent membrane that acts like a lens where its surfaceconvexity or concavity can be controlled by a mechanically orelectronically by compressing or decompressing a deformable lens by anon-deformable object, such as a magnetic plate or a piezoelectricsystem.

In one or more embodiments, the fluidic lens of the light field camerais a soft flexible lens which can be deformed electronically by applyingpressure on the lens to increase or decrease the refractive power of thelens.

In one or more embodiments, the fluidic or tunable lens of the lightfield camera is combined with a solid lens, thereby creating a hybridlens providing a minus or plus addition to the tunable lens.

In one or more embodiments, the fluidic light field camera shutter actslike a lid that covers the lens surface and is moved only up and down,or in horizontal direction to open or close the shutter, while in anopen phase, the lens is exposed or is bulged outward permitting a widerfield of view than the standard shutters provide.

In one or more embodiments, the tunable light field camera can bemonocular or binocular camera and the images can be used in augmentedreality (AR) or virtual reality (VR). The combination of two or moretunable field cameras can create 360 degrees in a 3-D format innon-dynamic and dynamic identity recognition.

In one or more embodiments, tunable light field camera with its softwareand algorithm can be used for dynamic recognition of an object, animateor inanimate, animal or human, where the dynamic changes can be observedin a 3-4 dimensional manner depending on which direction one likes toobserve the subject or a person which now can be presented with AR/VRglasses or subtracted to provide information about the changes that hasoccurred during the dynamic observation.

In one or more embodiments, the tunable light field camera can be usedfor producing videos in augmented reality (AR) or virtual reality (VR)and can be used with appropriate AR/VR glasses while simplifying theobservation of the image since the camera provides a focal point for anyarea which is being observed, thereby eliminating potentially the needfor convergence while seeing the object monocularly or binocularly.

In one or more embodiments, the combination of a tunable light fieldcamera and augmented reality (AR) or virtual reality (VR) can be used inthe medical field to virtually recognize the growth of a lesion (e.g., atumor) or the area of damage caused by a disease or trauma, etc. In thisapplication, one will be able to see the structure in three and fourdimensions during an invasive procedure, such as surgery, or a minimallyinvasive procedure through an endoscope, or a completely non-invasiveprocedure such as thermotherapy, using laser, focused ultrasound, or analternating magnetic field along with antibody coated magneticnanoparticles administered intravenously to seek the tumor cells andmake them subsequently visible by CT-scan, ultrasound, or MRItomography, or using the tunable light field camera and VR during theradiation therapy, laser therapy, or focused ultrasound, etc.

In one or more embodiments, the tunable light field camera and VR can beused by architects to see a building in 3D-4D format before it is builtand its structural stability is examined with augmented reality (AR)glasses or virtual reality (VR) glasses.

In one or more embodiments, using the tunable light field camera alongwith augmented reality (AR) or virtual reality (VR), architects are ableto see a building in 3-4D format before it is built and its structuralstability is examined with AR/VR glasses (e.g., in AR/VR during anearthquake, hurricane, tornado, etc.) produced in vitro to evaluatestability of a high-rise building, bridge, airplane, or any otherstructure, etc. or any area that requires 3-D, 4-D, or 5-D observationand planning (e.g., airline pilots to see their surrounding area toavoid an aerial crash by another plane or collision with birds, etc.).

In one or more embodiments, the light field camera is used as a part ofa dynamic facial recognition system for patient identification andverification, as described above. Advantageously, the tunable lens, whencombined with the other components of the light field camera, offersprecise focusing using the microlenses, nanosensors, and computer toanalyze numerous focal points that can be reconstructed mathematicallyusing specific algorithms implemented by the data processing device(i.e., the computer of the dynamic imaging system). Also, the dynamicimaging system may use the computer to verify or identify variouschanges that happen during the change in physiological function of aperson's facial expression (e.g., smiling or frowning) as acomputer-generated digital 2D or 3D image or video frame records thedynamic changes of a structure, such as a face, mouth, eyes etc., andthe computer analyzes and compares the biometrics as a dynamicphysiological fingerprint with existing data of the same image. Thecomputer algorithm analyzes the changes in the relative position of apatient's face, matching points and directions, and compressing the dataobtained during the process using dynamic recognition algorithms. Oneexemplary technique employed is the statistical comparison of the firstobtained values with the second values in order to examine the variancesusing a number of means including multi-spectral light that looks at thevarious physiological changes of the face. Mathematical patterns of thedigital images and statistical algorithms are capable of demonstratingthat the images obtained initially and subsequently belong to the sameperson, etc. For example, the computer may compare the images usingknown techniques, such as elastic bunch graph matching, face recognitionusing a Fisherface algorithm, face recognition using dynamic linkmatching, etc. Also, these techniques may be employed by the dynamicimaging system for exploratory data analysis to predict the changes ofthe image (e.g., aging of a face or tumor growth, etc.). For example, adynamic analysis of the growth of a tumor may be performed using thedynamic imaging system described herein (e.g., by analyzing the increasein the surface area or the volume of the tumor). That is, using thesystem described herein, the volumetric changes of a tumor or lesion iscapable of being measured over a time period by software subtractionalgorithms of the system as explained below, and then transmitted to thetreating physician. In addition, the dynamic changes in a portion of apatient's body may be compared with existing data for diagnosis ordifferential diagnosis so as to track and analyze trends in diseaseprogression or disease improvement against baseline data for managementof diseases. The dynamic imaging system described herein may beconfigured to track changes in the disease process (e.g., diabeticretinopathy, another retinal disease, brain, spinal cord vertebrae,prostate, uterus, ovarian, intestine, stomach, extremities, lung, heart,skin disease, eczema, breast cancer, a tumor in the body, etc.) over aperiod of time so that the disease process is capable of being monitoredby a physician. Also, follow-up images may be acquired using X-ray,CT-scan, positron, MRI, ultrasound, or photoacoustic imaging, etc.

In one or more embodiments, the body portion of the person being trackedby the imaging device comprises a tumor, lesion, retina, lung, or heart,and the dynamic imaging system is configured to dynamically analyze thegrowth of the tumor or lesion over a period of time by tracking thedisplacement of the tumor or lesion in a two-dimensional orthree-dimensional manner over the period of time.

In one or more embodiments, a grid is projected over the area ofinterest in the dynamic facial recognition system to clearly define theposition of each pixelated area of the face to compare it with the imageor displacement that has occurred in the process, and to superimposeimages on each other by the computer executing a dynamic subtractionsoftware algorithm to demonstrate the degree of the change and its trendduring the displacement or change, thereby presenting it as a subtractedimage when using a multispectral camera capable of analyzing the widespectrum of the wavelength (images) including the infrared or nearinfrared wavelengths and analyze them by the software of the camerarapidly presenting it as 2-D or 3D images (refer to FIGS. 18a and 18b ).

FIGS. 18a and 18b depict a grid 506 that has been projected over thefacial area of a person 500 in order to track dynamic changes in thelips 508 of the person 500. In FIG. 18a , the positions of two exemplarypoints 502, 504 are being used to track the dynamic changes in the lips508 of the person 500. In FIG. 18b , the positions of two exemplarypoints 512, 514 are being used to track dynamic changes in the lips 508of the person 500 while the person 500 is speaking the letter “O”. Asshown in FIG. 18b , as the letter “O” is spoken by the person 500,grooves 510 are formed in the skin of the facial area surrounding thelips 508.

When the system is equipped with a multispectral camera, themultispectral camera may be used to obtain photos either in the visiblespectrum or infrared to low infrared light spectrum working as athermographic camera, seeing deep inside the skin to recognize thestatus of the circulation under the skin. The infrared patternrecognition capabilities of the system record psychological functionalchanges occurring under the skin (such as an increase or decrease in thecirculation due to sympathetic activation-deactivation) together withdynamic changes, which are not achievable with a camera having onlyvisible light capabilities.

In one or more embodiments, the visible spectrum provides informationfrom the surface structure during the dynamic facial changes caused bydeliberate activation of the facial muscles producing skin groovesaround the mouth and the eye demonstrating the physical aspects ofchanges in a person being photographed in a two-dimensional and/orthree-dimensional manner.

The computer software of the system analyzes both of the aforedescribedfacial changes and presents them as independent values that can besuperimposed mathematically by the computer's software creatingsubtraction data indicating the changes that have occurred serving asthe initial face interrogation data. This subtraction algorithm may beused subsequently for recognition of a face, body, a tumor located onthe surface of the skin or inside the body imaged to recognize theextent of changed values in two or three dimensional format. The dynamicimaging system described herein may also be used also along withstandard imaging systems such as X-ray, CT-scan, MRI, positron, OCTimaging, photoacoustic imaging, thermoacoustic imaging, or optoacousticimaging to record changes occurring in the images and its temperatureafter treatment over a time period. Because the images are pixilatedbits of information that can be recorded live (e.g., for comparing animage before surgery and after the surgery), the images can besubsequently subjected to analysis with the subtraction algorithm of thecomputer software to decide, for example, if a tumor is completelyremoved or not.

In one embodiment, the algorithm and the software of dynamic imagerecognition system can be used to analyze the changes that can occurusing artificial intelligence (AI) with a standard image technique suchas X-ray, CT-scan, MRI, ultrasound, photoacoustic tomography,microscopy, optical coherence tomography (OCT), or confocal microscopy,etc. to diagnose early minute changes that occur in the body or tissueas a result of a disease process, such as macular degeneration,diabetes, heart or vascular disease, urogenital diseases, liver, kidney,or gastrointestinal diseases, extremities and joint diseases, skindiseases, Alzheimer's disease, Parkinson's disease, or othercerebrovascular diseases.

As another example, when the subject frowns, skin capillaries becomecollapsed and folded, thereby reducing the blood flow through thecollapsed and folded capillaries and the heat that is detected by themultispectral camera. The subtraction algorithm provides the exactchanges in facial structure during the frowning of the subject.

In one or more other embodiments, the subtraction algorithm executed bythe computer presents only the subtracted image and its mathematicalweight or value, and compares it with the previously obtained subtractedimage of the same structure or face to verify the identity the personand compare the initial values and the extent of the changes between thefirst and second captured images, and the trend of the changes that haveoccurred after displacement of the surface or a structure of interest,such as a tumor dimension over time and its growth trend.

In one or more embodiments, the before or after image subtraction may beperformed during the imaging of a tumor (e.g., a brain tumor) usingcontrast agents or antibody coated nanoparticles to demonstrate thedegree of involvement of the brain and the effect of surgery or therapyon the 3-D dimension of the tumor indicating whether the tumor is ableto be removed or treated completely or not. In one or more embodiments,the same computer algorithm is used in machine vision, robotic vision,or drone vision to examine the before and/or after effect of an actionover time and predict the trend.

In one embodiment, in dynamic facial recognition, a conventionalexisting mathematical algorithm is not used to compare two static imagesand conclude the possibility or probability of them being the same, butrather the computer of the present system is specially programmed tocompare two sets of dynamic images, which are composed of one static andone dynamic, that add significantly more information by dynamic changesin a two-dimensional and/or three-dimensional manner that have occurredas a result of displacements of various points (e.g., in the face of theperson) and the trend of the changes obtained by the computersubtraction of the dynamic changes, where to this, two significantvalues that augment the weight of the computer-generated image bysuperimposition of the obtained data or pre and post dynamic images forsubtraction analysis by adding the voice or sound waves recordedsimultaneously or superimposed over the dynamic facial values, andfinally confirming the data with dynamic fingerprinting that has twoadditional components of finger print analysis, and multispectralphotography before and after pressing the finger over the transparentglass, and collapsing the capillaries of the hand or the finger thatprovides practically a complementary algorithm for nearly infallibleidentity recognition data of a person within a timeframe of 2½ secondsor less. For example, FIG. 26 depicts a graphical representation 636 ofthe correlation between the mouth configuration and the sound wavegenerated by a person while speaking the words “I love you”. The firstsection 638 of the sound wave corresponding to the speaking of the word“I” produces an open mouth configuration 644 with wrinkles on the lipsof the mouth. The second section 640 of the sound wave corresponding tothe speaking of the word “love” produces a semi-closed configuration 646of the mouth where the lips touch the teeth. The third section 642 ofthe sound wave corresponding to the speaking of the word “you” producesa semi-closed, generally round configuration 648 of the mouth.

In one embodiment, the dynamic facial recognition technology describedherein uses a software program that not only subtracts the data obtainedin the first session from the data obtained in the second session, butsimilarly through its subtraction algorithm, compares the sound wave ofa person at each examination setting to show the difference or thetrend, along with final confirmation of the data with dynamicfingerprinting technology of pressing a finger or palm of the hand overa transparent glass which is photographed before and after pressing thefinger/palm with multispectral light waves to demonstrate the finger'sor palm's circulation pattern, or its collapse after pressing it overthe transparent glass, in addition to the ridges of the skin, folds,minutiae, etc. in a two-dimensional and/or three-dimensional manner fordynamic identity recognition.

The technology described herein demonstrates a way to subtract theinformation of a dynamic change mathematically from a dynamicrecognition data of not only the face, but also extremities or a movingperson or variation of the facial folds measured by a multispecteral orhyperspectral camera or color changes of the collapsed capillariesobserved after pressing the finger or palm, or changes of the facialcapillaries during an interrogation, or observation of a joyful or sadmovie, etc., which all add value to the correct identification of aperson.

In one embodiment, the subtraction algorithm may also be used to predicta trend using one of the existing mathematical algorithms describedbelow, and recognize changes or the trend for evaluation of dynamicimaging, and another dynamic imaging related to the first set, such asadding a value or one dynamic facial recognition with another onecreated by pressing of the face to bleach the facial capillaries orinducing an artificially induced element that changes the facial folds(e.g., created by frowning to that of the voice and dynamic changes ofthe mouth by speaking a vowel or smiling, etc.) in a two-dimensionaland/or three-dimensional manner.

In one embodiment, the accuracy of the dynamic facial recognition systemmay be augmented with the use of another existing mathematicalalgorithm, such as an artificial neuronal network, interval finiteelement analysis, fuzzy logic variations, machine learning, rough set,Sorties Paradox, vector logic, etc.

In one embodiment, the camera of the dynamic facial recognition systemis equipped with software for machine learning, with a neuronal networkand medical imaging or perception, natural language, speech recognition,big data decision-making of the changes that occurs in a dynamic formator anthropology software, or artificial intelligence capable ofreconstructing a complete image from a part of the image or informationthat is obtained from a person.

In one embodiment, the person being imaged or photographed is an activeparticipant in the process that implies the person in giving his consentby participating in the process and his photograph is not taken withouthis permission randomly by a camera located in a location.

In one embodiment, standard facial recognition may be used with atelemedicine system despite its shortcoming of the person's recognitionand the fact it does not provide or imply a person's consent, by using arecording to record the person's consent for telemedicine communicationby asking the person to indicate his or her consent by responding to thequestion if she or he would give permission to discuss his or her healthor other issues by indicating “yes” or “no”, or any other meansvoluntarily while being recorded by a camera or voice recognitionsystem, etc.

In one embodiment, the person writes “yes” or “no” while the computerrecognizes the handwriting with the handwriting recognition, or theperson presses over an area of the monitor of a computer where thecomputer has selected indicating “yes” or “no” where the computer canrecognize the person from the minutiae of the tip of her or his fingeralong with dynamic recognition of the capillaries, or simultaneouslyobserving the dynamic changes of the person's face, etc. (i.e.,recording two or more independent tasks for recognition).

In one embodiment, the dynamic recognition is used in telemedicine, andone of its function is to secure a person's consent in telemedicine thatinvolves asking a person for his consent to take a photo or dynamicimaging from him or any of his or her body part and obtaining his or herconsent verbally or by head or hand or eye finger motion, etc., oradding his fingerprint to a surface of the computer or a smartphone thatcan record it in a 2D or 3D form or in a dynamic format with imaging thechanges in the fingertip or the organs or capillaries or using a sensorto simultaneously measure the pulse rate, etc. of the person, while thevoice recognition software records the sound waves of a patient in hisnative language or another language for future verification, and thesound waves are recorded, stored and encrypted.

In one embodiment, the dynamic image recognition is used in combinationwith computer artificial intelligence (AI) software and a camera thatmimics human recognition by learning and solving problems includingidentity of a person or a structure or a lesion, its dynamic changes anddata searching, calculating, sensing the environment in depth orsensing, the social intelligence using linguistic, psychology,mathematic and neuroscience to create new images and recognize theemotional changes of a person or animal.

In one embodiment, the dynamic image recognition system is capable oflearning algorithms that can recall, learn, and infer changes inrecognizing a person, lesion, etc. in any image presented with standardphotography or hyperspectral imaging, X-ray, MRI, or CT-scan, microscopyor confocal microscopy or OCT, etc. using hierarchal temporal memorysoftware.

In one embodiment, the dynamic image recognition system uses themathematical means of fuzzy logic or vagueness in a complex situationfor recognition of difficult images to differentiate certain persons(e.g., twins, etc.), along with dynamic identity recognition and machinelearning, speech recognition, to differentiate the effect of theenvironment on the person or animal or any other living structure.

In one embodiment, the dynamic identity recognition system uses theartificial intelligence (AI) with various existing softwareinstructions, algorithms, such as dimensional reduction, structuralprediction, anomaly detection, artificial neuronal network, parallelconstraint satisfaction, or parallel distributed processing, etc.

In one embodiment, the dynamic identity recognition is used with roboticmachine vision for macroscopic or microscopic differentiation of atleast two similar or dissimilar objects, the position, and 3-D format inorder to perform a task precisely as instructed by the software.

In one embodiment, the tunable field camera can have a tunable lens orthe one fluidic lens.

In one embodiment, the tunable lenses are tunable lens withshape-changing polymer lenses (i.e., Optotune®), liquid crystal lenses,electrically tunable lenses, electrowetting lenses (Varioptic®), or thefluidic lenses described by Peyman in U.S. Pat. Nos. 7,993,399 and8,409,278, etc.

In one embodiment, the tunable light field camera can be an integralpart of the laser coagulation system or be provided separately as acamera.

In one embodiment, a tunable light field camera or other field camerascan be used with the tele-laser system or separately, or they can alsobe used as a part of the facial recognition system as a dynamic facialrecognition system. Tunable light field cameras offer a precise focusingconcept using the microlenses, nanosensors, and a computer to analyzevarious focal points that can be created mathematically using specificalgorithms.

In one embodiment, the tunable light field camera uses the computer toverify or identify various changes that can happen during the change inphysiological function of a person, such as smiling or frowning, as adigital 2 D or 3-D image or video frame recording the changes andanalyzing and comparing the biometrics as a dynamic physiologicalfingerprints with the existing date of the same image (e.g., a face,etc.) using mathematical dynamic pattern recognition. The computeralgorithm analyzes the changes in the relative position of a patient'sface by matching points and directions and compressing the data obtainedduring the process using dynamic recognition algorithms.

In one embodiment, the dynamic identity recognition uses a geometricalgorithm to distinguish the features of a person or a lesion, etc., andphotometric analysis provides precise statistical data from the imagesto compare the data template e-variances.

In one embodiment, a retinal scan provides not only the identity of aperson evaluating a unique pattern of a person, but can be used toanalyze the changes in the thickness of the retinal structures, alongwith OCT and analyze the changes of the retinal capillaries and retinalpigment epithelium, etc. using the known principal component analysis orlinear discriminant analysis (LDA), generalization of Fisher's lineardiscriminant, pattern recognition, and machine learning to provideprognostic information (e.g., in age related macular degeneration ofdiabetic retinopathy, glaucoma, and other diseases causing retinaldegeneration or abnormal blood vessels).

In one embodiment, in dynamic identity recognition (DIR), one uses 3-Dimages that are created for skin texture analysis by projectingstructured light on the skin, mucosa or retina, etc. (e.g., for asecurity system at an ATM).

In one embodiment, the computer modifies the 3-D image of a person takeninstantaneously or an existing image and modifies it with its artificialintelligence (AI) system for a search of similar images in its file orthe computer can modify the person or animal image by adding orsubtracting certain variable components to the person, such as addingdifferent types of hair including color, etc., or remove the hair,changing the facial hair, adding or subtracting earrings, changing theskin color or the color of the iris, adding or subtracting glasses, etc.By confirming the stable points in an image (e.g. distances between theeye, nose, mouth, ear, etc.—see e.g., stable features 614 in FIG. 22) toconfirm the identity of the person being examined, these tasks can beperformed rapidly by a processor.

In one embodiment, the dynamic identity recognition system can add orsubtract, using software and artificial intelligence (AI), a 3-Ddistinct feature after it has checked a person with the dynamicrecognition system (i.e., before and after dynamic images and computergenerated modifications). This computer software has the ability toartificially create an instant match with the person that is beingexamined by creating hundreds or thousands of similar images (e.g., byadding hat or glasses or changing the hair color of the subject and lookfor comparison in its previously stored memory). As shown in FIGS.20-25, the specially programmed computer of the dynamic identityrecognition system is able to eliminate the beard or hair of a personwhile evaluating other stable features or dynamically changed featuresof the same person, including the voice of the person, etc. For example,in FIG. 20, an initial image of a person 600 depicts the person withhair 602, a moustache 604, and a beard 606. In FIG. 20, the mouth 608 ofthe person is in a closed configuration. Turning to the image 610depicted in FIG. 21, it can be seen that a dynamic change in theappearance of the person has taken place. More specifically, a dynamicchange in the lower section of the person's face has occurred (i.e., themouth configuration 608′ is different from that of FIG. 20, resulting inenhanced folds on the face and visible teeth), while the midsection ofthe person's face has remained generally stable without any significantchanges.

In one embodiment, the software of the dynamic identity recognitionsystem can change the color of the skin, hair, or beard of the subjectusing all possible colors of the light spectrum, or adding hats or hair,toupee hair pieces or mustaches, or earring(s), etc. and/or convertingthe face of the person or patient to an emotional face showing, beingsad, being happy, or being in pain, etc. using the appropriate existingfolds and skin capillary color, fold compression enhancing or decreasingthe capillary circulation creating specific color intensity specific tothe emotional and physiological state of the subject's feelings, such aswell-being, using an infrared (IR) and visible light system in 3-D alongwith AI, AR, and VR software, using the voice recognition software forthe subject's voice and variation of sound frequency and amplitude torecognize the physiological state of the subject. For example, in FIG.22, a computer-modified image 612 of the person is depicted where thehair 602 of the person has been removed, thus resulting in a bald head616. Also, the facial hair of the person has been removed, therebyresulting in a hairless chin 618. The computer-modified image 612 inFIG. 22 enables the dynamic identity recognition system to bespoof-proof by enabling the system to still accurately identify theperson even if the person shaves his head, and removes his facial hair.In the case of such changes in the appearance of the person, the dynamicidentity recognition system is still able to accurately identify theperson by focusing on the stable features 614 of the person. Turning tothe image 620 depicted in FIG. 23, it can be seen that a dynamic changein the appearance of the person has taken place. More specifically, adynamic change in the lower section of the person's face has occurred(i.e., the mouth configuration 608″ is different from that of FIG. 22,resulting in enhanced folds on the face and visible teeth), while theupper section and midsection of the person's face has remained generallystable without any significant changes. In FIG. 24, anothercomputer-modified image 622 of the person is depicted where glasses 624have been provided on the person, the iris color 626 of the eyes of theperson has been changed, and an earring 628 has been added to one of theears of the person. In order to make the dynamic identity recognitionsystem spoof-proof, other computer-generated anatomical and cosmeticvariations can also be made to the person, such as changing the skincolor of the person, adding a hat on the head of the person, addinglipstick to the person, changing hair color, changing facial hair color.These computer-generated changes to the image of the person enhance thelikelihood of an identity match by the system even when the personalters his or her appearance (e.g., by shaving his head, shaving hisfacial hair, etc.).

In one embodiment, the dynamic identity recognition system can alsosimultaneously recognize via the software executed thereby the emotionalchanges that are associated with the existing folds and capillaries ofthe skin of a person's face that are enhanced by the emotional changesand can be subtracted from the existing prior images of the person orthe patient, etc. In FIG. 25, another computer-modified image 630 of theperson is depicted where an artificial change in the appearance of theperson has occurred while the person is speaking one or more words(i.e., represented by the sound wave 634 in FIG. 25). As shown in FIG.25, the computer-modified image 630 simulates a dynamic change in thelower section of the person's face while the person is speaking the oneor more words (i.e., the mouth configuration 608′″ is different fromthat of FIG. 24, resulting in enhanced folds on the face and visibleteeth). Also, in FIG. 25, simulated hair 632 has been added to theperson to increase the probability of an identity match if the persongrows hair after being bald.

In one embodiment, the dynamic identity recognition system can diagnosethe emotional changes by the use of its camera's imaging in the visiblespectrum or in the infrared (IR) spectrum.

In one embodiment, the computer of the dynamic identity recognitionsystem can record and subtract the emotional changes instantaneouslyinduced by conditions, such as anger or smiling, sickness, pain, orasking a question or remembering a past experience or trauma, etc.

In one embodiment, the computer of the dynamic identity recognitionsystem presents the before and after images and the subtracted one(e.g., from the original person's file to diagnose the physiological oremotional state changes of the subject).

In one embodiment, the dynamic identity recognition system eliminates orcreates spoofs for potentially creating a semi-virtual representation ofthe person with arbitrarily changing certain characteristics (e.g., thehair for matching the images of the subject for comparison purposes, orrecognizing the dynamic emotional changes of the subject both useful inthe medical field for diagnosis of an existing psychological conditionor use in a security system recognizing the person's response to, forexample, a question or a joke recognizing the physical and emotionalchanges in the images in 2-3-D format—see e.g., FIG. 25).

In one embodiment, all photos can be observed with specific glasses orgoggles as AR or VR. Since, the images are taken in 3-D, they can beobserved in 3-D and from any direction and volumetric analysis of theobject (e.g., head, nose, ear, chin, or a tumor, etc. can be compared oranalyzed).

In one embodiment, the computer of the dynamic identity recognitionsystem can access any library of images for obtaining existing priorimages or information, or similar images including clinical ones such asX-ray, MRI, CT-Scans, or photographs, etc. for analysis etc.

In one embodiment, a computer with dynamic identity recognitioncapabilities and artificial intelligence (AI) can generate from a personor animal a present image and a new computer-generated older or youngerperson's secondary image as long as certain elements or similarities arediscovered in the initial image to identify in the 2 to 3 dimensionalimage of the present image using AI and artificial neural networks (ANN)to create a new secondary image to build on the bases of unchanged partsof the present image (e.g., stable features 614 in the image 612 of FIG.22), and alternative images with the same basic characteristics foridentification using a software similar to those utilized inanthropology, to discover a whole structure from only a part of theimage or generating identifying characteristics from the examples(present image) that it processes.

In one embodiment, the software and algorithm of dynamic identityrecognition (DIR) and artificial intelligence (AI) can be used in therecognition of a person or an animal, or a person lost because of amental disease, such as Alzheimer's disease, or in a forensic situation.

In one embodiment, the software algorithms of dynamic identityrecognition (DIR), artificial intelligence (AI), and artificial neuralnetworks (ANN) with distributed computing and deep learning or thebi-directional and multi-dimensional long short-term memory (LSTM) orconvolutional neural networks (CNN), etc. can be used in systemidentification or control in trajectory prediction in microscopic ormacroscopic fashion, general game playing, 2-3 D pattern recognition ofinanimate objects or live humans, animals and other living things,signal classification, voice recognition, winking, gesture, or otherfacial expressions, etc., word or sounds expression of good or bad,handwriting recognition, medical, recognition of a disease process suchas cancer, before and after surgical procedure for tumor or inorthopedic surgery etc., visualization, microscopy, recognition, such ascells, tissue samples, bacteria, fungi, viruses, in swab, biopsy or intissue culture with simultaneous quantification.

In one embodiment, the dynamic identity recognition system utilizescomponents of artificial intelligence (AI) including training,validation with image and data, combined with a technical network, suchas a convolutional neuronal network (CNN). The training includesrecognition of obvious characteristics (e.g., such as face, eyes, nose,mouth, etc.) requiring focused images in 2D to 3D format or dynamicallychanged images that provide additional data along with voice recognitionwith deep learning algorithms such as CNN, VGGNet, AlexNet, InceptionV4, DenseNe, and ResNet.

In another embodiment, in a clinical setting, the dynamic identityrecognition system utilizes components of artificial intelligence (AI)in evaluation of a disease process in addition to what is describedabove for identity recognition of an individual. Artificial intelligence(AI) requires a standard or normal reference and means of comparison byhuman experts along with data collected from a large number of subjects.The machine learning or deep learning in artificial intelligence (AI)includes data obtained from clinical presentation, images and additionalmultimodal data, such as laboratory information used with deep learningalgorithms, such as neuronal computational layers available as CNN,VGGNet, AlexNet, Inception V4, DenseNe, and ResNet.

In another embodiment, the multispectral or hyperspectral camera is usedto obtain both spectral and spatial data from the images before andafter the dynamic changes have occurred on an external area of thesubject. In one embodiment, the characteristics and changes of a movingobject is recorded and analyzed providing simultaneous imaging,processing, and evaluation by the software of the processor of thecamera, thus sensing the simultaneous changes that are happening and thetrend of the changes (e.g., on the surface or below the surface of theskin or a subject being photographed). In one embodiment, the dynamicimages obtained from the surface can be combined with the informationobtained by the ultrasonic unit of the laser system to provideadditional information from a deeply located, internal body structure,such as bone, joints, or a moving object, etc. In one embodiment of theremote laser system, the electronically obtained images are combinedwith CMOS image sensors (e.g., analyzing a subject's fingerprint cangive information on the blood flow of the fingertip before or afterapplying pressure with the finger that collapse the fingers skincapillaries and the changes may be analyzed in real-time).

For example, a dynamic fingerprinting and/or dynamic hand recognitionsystem will be described with reference to FIGS. 19a-19c . Initially, asshown in FIG. 19a , a finger 516 containing numerous ridges and folds isplaced on a surface of transparent glass 518 so that the finger is ableto be imaged (i.e., photographed or videoed) with an infrared spectrumand/or visible spectrum of a field camera, multispectral camera, orhyperspectral camera 520 of the remote laser system. Rather than simplyimaging the fingertip in FIG. 19a , the entire hand of the person alsomay be imaged using the camera 520. The system of FIG. 19a may alsorecord the color of the fingertip or hand of the person prior to itsplacement on the surface of transparent glass 518 together with thefolds and ridges of the fingertip and hand.

Then, turning to FIG. 19b , a finger 522 is illustrated prior totouching the surface of transparent glass 518 and being imaged by thefield camera, multispectral camera, or hyperspectral camera 520 of theremote laser system. FIG. 19b depicts the fingertip or ball of thefinger 522 with its circulations, ridges, and minutiae, which are ableto be imaged using the camera 520 for highly reliable identification ofthe person. The infrared spectrum of the camera 520 is able to recordthe warm circulation of blood through the fingertip or ball of thefinger 522. FIG. 19c shows the ridges of the fingertip of the finger522, but centrally, the capillaries of the finger 522 are collapsed atthe area where the fingertip or finger ball is touching the surface oftransparent glass 518, which indicates that a live person is beingrecorded. In FIG. 19c , where the capillaries collapse at the ball ofthe finger 522, the finger 522 becomes devoid of circulation (thusturning red in color) at the site of compressed area compared with theprevious image; these changes are present in infrared photography.Advantageously, the system of FIGS. 19a-19c preserves the folds in thefinger 522 or, if the whole hand is placed on the glass 518, the foldsin the whole hand. In the illustrative embodiment, all of thisinformation is recorded before and after placement of the finger 522 orhand on the glass 518, and the changes are subtracted to obtain theverification of the person's identity, and the physical andphysiological changes that have occurred are analyzed to recognize andverify the person's identity.

In one embodiment, the person's finger, hand, or other extremities arevideoed to create more complete information regarding the identity ofthe person or the area of the body recorded for future dynamiccomparison, etc. In addition, a voice instruction may be included in thesystem of FIGS. 19a-19c in order to ask the person to press harder, orloosen up his or her finger, so that the degree of the capillarybleaching is confirmed, etc.

Also, in one or more embodiments, three-dimensional dynamic data isobtained either from multiple cameras or from the mathematical analysisof the obtained digital data from the light field camera system.Advantageously, the rapid variation of the light field camera eliminatesthe problem seen with moving objects that interferes with a good staticfacial recognition. Additional data can be obtained on the skin and itschanges during the physiological dynamic imaging with an infrared cameraor multispectral camera.

In addition to use in the remote laser treatment system 50′ describedabove, the dynamic imaging system described herein may also be used forother applications, such as for security system applications, use inHoloLens applications, use in telemedicine applications (e.g.,tele-imaging or telediagnostic systems), and use in other patientapplications, such as looking at the growth of various lesions over timeand aiding with diagnosis and differential diagnosis. Other applicationsfor the dynamic imaging system described herein may include specificapplications involving hospitals for patient security and operating roomapplications, customs department, airports, the state department, police(e.g., for investigative analysis of the crowd or identification aperson), the military, FBI, CIA, various other governmental agencies,and banking institutions (i.e., for account and ATM identification). Thedynamic imaging algorithm may also be used in robots, drones, inagriculture applications, or in military applications. The dynamicimaging system may also be useful at stadiums of competitive sportingevents, such as football, soccer, basketball, and hockey stadiums, andat other venues involving large gatherings of people for political ornon-political causes, which often require some permission to guaranteethe privacy and safety of the people present at the event. In addition,the dynamic imaging system may be used in smartphones for remoterecognition, and in home security systems, etc.

In one or more embodiments, the dynamic facial recognition systemdescribed herein may replace previous verification/identificationsystems used in personal life, such as passwords IDs, PINs, smart cards,etc. The dynamic facial recognition system also has unlimitedapplications in personal security, identification, passports, driverlicenses, home security systems, automated identity verification at theairports, border patrol, law enforcement, video surveillance,investigation, operating systems, online banking, railway systems, damscontrol, medical records, all medical imaging systems, and video systemsused during surgery or surgical photography to prevent mistakes in theoperating room (e.g., mistaking one patient for another one, or oneextremity for the other). The dynamic imaging system described hereinmay also be used for comparative image analysis and recognizing thetrends in patients during follow up analyses and outcome prediction. Thedynamic imaging system described herein may also be used with otherimaging modalities, such as X-ray, CT-scan, MRI, positron, photoacoustictechnology and imaging, ultrasound imaging, etc. Further, the dynamicimaging system may be used for image and data comparison of close orremotely located objects, mass surveillance to document time relatedchanges in the image, and/or recognizing a potential event and itstrend.

Laser-Imaging System with Global Positioning System (GPS) Capabilities

In another further embodiment, the laser-imaging system 50′ may beprovided with GPS-based identification of the laser treatmentapplication site. With reference to FIG. 17, it can be seen that thelocal control systems disposed at local sites 51 a′, 51 b′, 51 c′ mayeach include a Global Positioning System (GPS) receiver unit 76 thatcommunicates with a cluster of GPS satellites 77. In particular, the GPSreceiver unit 76 may receive radio signals from a cluster of satellitesin orbit around the earth so as to be capable of automaticallyindicating the locations of the local control systems disposed at eachof the local sites 51 a′, 51 b′, 51 c′. As such, the specific locationof each local control system is able to be communicated to the commandcomputer 59 b of the central control system 58 via the Internet-basedconnection 60. In addition, as shown in FIG. 17, the central controlsystem 58 may also include a GPS module 78 that communicates with thecluster of GPS satellites 77 and the GPS receiver units 76 at the localsites 51 a′, 51 b′, 51 c′.

In the illustrative embodiment, as shown in FIG. 17, the GlobalPositioning System (GPS) is included in the laser delivery system (i.e.,at local sites 51 a′, 51 b′, 51 c′) located away from the control siteor doctor's office (i.e. at the remote site), providing informationlogged in its memory on where the laser system is located at any time orwhere the system might have been transported from the time of theconstruction to where it is moved in real time or is recorded in thesystem and can be retrieved.

In one or more embodiments, smartphones, computers, or the remotecontrolled laser systems at local sites 51 a′, 51 b′, 51 c′ are equippedwith GPS so that the system starts sending the information on theposition of the device as soon as the laser delivery system, a phone, ora computer is turned on.

In one or more embodiments, the GPS capabilities of the system areautomatically switched on so that when the system sends a signal, it isreceived by the laser control system along with photographs or video ifthe 3-D camera is activated at the doctor's office located away from thelaser system. When the system is turned on, the system automaticallytransmits a signal about its position at all times, which can bereceived by the control system while a data pusher tracks the device.

In one embodiment, the high definition system permits you to zoom intothe video and maintain the clarity of the image at any point.

In one or more embodiments, the laser delivery system or a smartphone ora computer may contain covert GPS trackers and the camera of the laserdelivery system identifies the patient, person, or people that arepresent along with the patient's medical history, or any other personalrecord, etc.

In one or more embodiments, the unit is programmed so that if the GPS istouched, or in any way manipulated, it will self-destruct (e.g., aftereither being exposed to the light, or mechanically handled). Forexample, piezoelectric nanoparticles may be used to create an electricalsignal that is deadly to the function of the GPS and render it incapableof sending a signal that is able to be recognized.

In one embodiment, the receiver's computer at the doctor's office firstindicates the position of the sender, then it evaluates and matchesparameters that indicate the identity of the location, by automaticallyturning on the cameras of the laser delivery system recording andsending out live stream of video as is the case, for example, with theSkype™ system. The system captures images and/or sound related to theenvironment, person(s), etc. and analyzes them using familiar facesand/or voices, such as the nurse's face and/or voice, and the necessaryphysiological functions and their changes (e.g., changes in the face,etc.) indicating the familiar person and the prior history.

The system may also send a distress signal, which initiates a remotesignal to the desired original place and automatically activates asignal to damage the laser delivery system and its software to preventits operation.

In one embodiment, the person or patient who is being examined ortreated is checked and his or her identity is verified before anycommunication between the remote system and the doctor's office isestablished by the data obtained and the software available at thedoctor's location prior to the system acknowledging that the connectionline is open to further interaction.

In one embodiment, the software at the doctor's location decides aboutwhether to permit the patient's site to communicate with the doctor,etc. by comparing or matching the present data with the previous datathat is available to the system.

In one embodiment, the doctor initiates communication by checking anyvariations and the trends in the changes of the physiological functionof the patient located at the remote area while the system evaluates andcorrelates it with available data.

In one embodiment, the remote laser delivery system always comprises acomponent that is in a permanently “on” position so as to connect thesystem to the GPS, which indicates the exact and detailed presentlocation of the remotely located system via a live GPS map, whilerecording automatically the time and the map if the unit has been moved.The GPS module 78 indicates any changes in the position of the unit onthe background map and/or generates images or other information for thedoctor.

In one embodiment, the camera system is turned on permanently andrecords the surroundings using its pattern recognition and/or 3-Ddisplay. As such, the system identifies the location and a person usingthe static and non-static or physiological changes that have occurred toidentify the person or persons or patient(s), etc., and transmits thedata to the doctor's site automatically using a 3D (stereo) camera witha head-up display.

In one embodiment, the camera at the remote laser delivery site, is adigital light field photography (DLFP) camera requiring microlenses thatcaptures the information about the direction of the incoming light raysor a plenoptic unit where the arrays of photosensors pixels are placedbehind the microlenses, and algorithms of the software are used toprocess the information obtained, as known in the art.

In one embodiment, the camera of the remote laser delivery system isequipped with a light field camera, such as Illum, CAFADIS, or roboticeye, that can provide information at different distances from the cameralocated in front or behind the focal point of the camera using softwareof the computer creating 3-D images, or stereo images using the softwareor graphics processing units (GPUs) or field programmable gate arrays(FPGAs).

In one embodiment, the doctor permits the system to analyze the data foridentification using the system software analyzing in the memory bankthe different normal versus abnormal variables, positive or negativevariables, and other associations so as to create a pattern ofsignificant association and create a coated color intelligentassociation of the past and present images.

In one embodiment, after verification of the person, the system permitsexamination of a lesion, surface, or internal structure with ultraviolet(UV) to infrared light, optical coherence tomography (OCT), ultrasound,or photoacoustic images to compare with the existing information fromthe patient and from other patients so as to provide potential diagnosisand options for therapy, while the system verifies the location,person(s) or patient(s) involved, etc. before proceeding with thetherapy or an action and before the doctor runs the simulation of alaser treatment, if needed.

In one embodiment, the system provides for remote imaging or patternrecognition for a patient, persons, a surface, mucosal marking, orlesion along with verification using multi-spectral imaging, OCT,ultrasound or photoacoustic imaging and physical and physiologicalanalysis to compare the surface lesions for shape pattern circulationthickness in 2 to 3 dimensions with the lesions or faces seen previouslyand existing in its file to diagnose a lesion, face, etc. and analyzethe status of the progression of a disease, aging, and provide a list ofoptions to be used in each case that can be complemented with theexisting worldwide experiences stored in the memory of the computer ofthe unit.

In one embodiment, the connection to another requested computer isrouted through the central location to verify the signal as a “friend”with all the patient's information or person's recognition live prior togiving the permission to be connected to another system elsewhere.

In one embodiment, all the process and personal recognition isautomatically stored at the central location and may be retrieved againby a doctor or a person already known by the system. The desired 2-D,3-D, or video images may be retrieved along with recordings of theconversations between the parties.

In one embodiment, the control system performs only one way recognitionof the person at the remote location. In another embodiment, for anextreme security sensitive and urgent communication, the control systemverifies the people or each person on both sides live from its existingfiles, and then opens the communication to establish a secure bilateralsystem, mostly for short and concise communications while the two-waysystem can initiate only from the doctor's office side, thereby addinganother layer of confidence or veracity to the information.

In one embodiment, the existence of the system is known to only thepeople with top security clearance that carry a specific computer andcell phone.

In one embodiment, the computer of the laser delivery system at both thecentral and the remote location, or the laptop, and cell phones arereplaced with new ones after the entire information is retrieved and thedata is cleaned.

Laser-Imaging System with Photodynamic Therapy (PDT) Capabilities

In another further embodiment, the laser-imaging system 50, 50′ may beconfigured to provide remote photodynamic therapy (PDT) for the patient.More particularly, in this further embodiment, the treatment laser ofthe laser generation system may be configured and arranged to performthe photodynamic therapy (PDT). In this embodiment, the PDT treatmentmethod utilizes the combined effects of a photosensitizer in presence ofoxygen and a light of a specific wavelength emitted by the treatmentlaser that is absorbed by the photosensitizer so as to create a chemicalreaction, thereby producing singlet oxygen and reactive species. Theseare in oxidative processes and are toxic to the tissue damaging thecellular mitochondria if the photosensitizer is located inside the cell.If the photosensitizer is located outside the cell in contact with thecell membrane, it damages the cell membrane (e.g., endothelial cellmembrane of the vessels). The damage to the endothelial cell wall of thevessels causes platelet aggregation, cloth formation, and closure of thevessel. Addition of an anti-vascular growth factor to the PDT enhancesdamage to the abnormal blood vessels produced by the tumor.

In this further embodiment, after inducing the physiological changesdescribed above and verifying the patient's identification for theprocedure, the site and location of the intravenous injection of a PDTphotosensitizer and/or a surface lesion is captured by the imagerecognition sensor 73 of the laser-imaging system 50′ and displayed tothe remote physician on the 3D display so that it can be verified by thephysician. Then, a combination of the photosensitizer and anti-VEGF,such as Avastin or other agents (e.g., anti-platelet derived growthfactors) is applied to enhance damage to a cancerous lesion, such asskin or mucosa. Also, in addition to the photosensitizer, an anti-cancermedication, an anti-cancer immunotherapy medication, such as Braf andMek targeted therapy, and/or a programmed cell death PD-1 immunotherapyto enhance damage to the cancerous lesion may be applied. After the PDTprocedure, the patient is advised by the physician to avoid sun exposurefor one week to prevent a sun radiation effect on other parts of thebody.

Also, in this further embodiment, after inducing the physiologicalchanges and verifying the patient's identification for the PDT procedureusing the 3D display, nanoparticles coated with a biocompatible polymerand a photosensitizer conjugated with a thermosensitive polymer, such aschitosan, etc., are applied to the cancerous lesion of the patient. Thenanoparticles may be metallic, non-metallic, synthetic, organic,non-organic, a hybrid, magnetic, paramagnetic, diamagnetic,supramagnetic, non-magnetic, and combinations thereof. The nanoparticlesmay be in the form of graphene-oxide quantum dots, graphene-zinc oxidequantum dots, graphene nanotubes, and/or carbon nanotubes. Thenanoparticles may contain a combination of two to three elements, suchas gold, gold-iron oxide, iron oxide, iron-zinc oxide, metallicnanoparticles, polylacticglycolic acid nanoparticles, ceramicnanoparticles, silica nanoparticles, silica crosslinked block polymermicelles, albumin-based nanoparticles, albumin-PEG nanoparticles,dendrimer attached magnetic or non-magnetic nanoparticles, etc. Thenanoparticles also may be in the form of dendrimers, micelles,fullerenes, quantum dots, nanoshells, nanocages, nanorods, nanotubes,nanowires, and combinations thereof. The nanoparticles may be formedfrom gold and silica, ferric oxide, ferric, cadmium sulfate, platinum,etc., and may be coated with polyethylene glycol (PEG), fatty acid,chitosan, or another biocompatible molecule and conjugated with a tumorantibody and a photosensitizer. The nanoparticles may be conjugated witha tumor antibody and a cell penetrating peptide (CPP) so as to enhancethe penetration of the photosensitizer in the tumor cells. Thephotosensitizer and/or nanoparticles, may be given intravenously,applied topically, or taken orally. Preferably, the anti-VEGF is appliedlocally to avoid the side effects of systemic administration, unless itis delivered as coated nanoparticles which are functionalized with anantibody to seek specifically a tumor cell. In general, thefunctionalized nanoparticles are taken up preferentially by the tumorcells and the growing proliferating endothelial cells of the abnormaltumor cells. When administered intravenously, the photosensitizer and/ornanoparticles may be configured to reach the desired area and attach tothe desired tumor cells by utilizing a monoclonal antibody, a polyclonalantibody, or aptamer-coated nanoparticles to seek out the tumor cells.Also, in this embodiment, functionalized pluralities of nanoparticles,which are coated with a photosensitizer, may be given topically orsystemically to a tumor located on or close to the skin or mucosa of thebody.

In this further embodiment, the nanoparticles may be in the form ofthree dimensional semiconductor devices using light or ultrasound energyto generate electrical energy to provide a photovoltaic effect. Inembodiments, the nanoparticle material may be ceramic, plastic, silicon;particles of crystalline silicon may be monocrystalline cells, poly ormulticrystalline cells, ribbon silicon having a multicrystallinestructure, nanocrystals of synthetic silicon, gallium/arsenide,cadmium/selenium, copper/indium/gallium/selenide, zinc sulfide, ironsulfide, iron-platinum, indium/gallium/phosphide, gallium arsenide,indium/gallium nitride, a nanocrystal, such as cadmium/selenium (Cd/Se)and a metal, e.g., a CdSe/Au nanometer-sized composite particle aspreviously described, particles of a variety of semiconductor/metal andsemiconductor/semiconductor hetero-junctions, e.g., particles based uponsemiconductor/metal hetero-junctions between group II-VI, IV, III-V,IV-VI, referring to groups of the periodic table, metal-oxide, ororganic semiconductors and a metal, and in particular those based uponSi/Au, GaAs/Au, InAs/Au, and PbS/Au hetero-junctions. The quantum dotsand/or semiconductor nanowires may also be biocompatible short peptidesof naturally occurring amino acids that have the optical and electronicproperties of semiconductor nano-crystals, e.g., short peptides ofphenylalanine. The particles can consist of both inorganic and organicmaterials, as previously described. In addition to being stimulated bylight and ultrasound energy, the nanoparticles may also be stimulated bymicrowave radiation, magnetic fields, alternating magnetic fields, andradiofrequency (RF) electromagnetic radiation.

Moreover, in this further embodiment, the application of the treatmentlaser for PDT is substantially applied in a continuous wave (CW) fashionat wavelengths that are absorbed by the nanoparticles and/or thephotosensitizer. The laser spot generated by the treatment laser may beapplied as a single spot covering the entire cancerous lesion andslightly beyond it. Alternatively, the treatment laser may be applied asa small spot, but using a painting or oscillatory technique bydisplacing an electrically controlled prism or a mirror, which islocated in front of the treatment laser beam, in an oscillatory manner.In this embodiment, the treatment laser of the laser generation systemmay produce a wavelength appropriate for the photosensitizer (e.g., awavelength of 405 nanometers (nm) to 420 nm, or 500 nm to 600 nm, or 635nm, a near-infrared laser from 600 to 1060 nm-1550 nm and more, up to aninfrared wavelength) which is absorbed by the photosensitizer andpenetrates the skin and mucosa up to a distance of 1 centimeter (cm) inthe tissue. In one or more embodiments, the treatment laser may comprisea powerful non-coherent light source from any light, such as a xenon ormercury lamp, rather a coherent light source.

Furthermore, in this further embodiment, the patient's cancerous lesionmay be pre-treated by the topical application of the photosensitizerand/or nanoparticles through the skin or mucosa, for a period of 1-30minutes depending on the thickness of the lesion to be treated. Thephotosensitizer may comprise one of aminolevulinic acid, methylaminolevulinate, Verteporfin, and riboflavin, etc. Advantageously,riboflavin is a non-toxic photosensitizer which still normally kills thetumor cells, and cross-links the collagen around the tumors so as tostrangulate and close their vascular supply as a result of itsphotodynamic effect. In one exemplary embodiment, a Verteporfinphotosensitizer preparation is conjugated with cell penetrating peptides(CPP) or activated cell penetrating peptides (ACPP) and polyethyleneglycol (PEG), etc. coated nanoparticles, and after the identity of thepatient is verified in the manner described above, the preparation isadministered intravenously to reach internally and externally-locatedtumor cells. In this embodiment, the size of the nanoparticles ismaintained below 10 nanometers (nm) to enhance their secretion throughthe kidney and urine. In other embodiments, the sizes of thenanoparticles are from 1 nm to 1000 nm. When the photosensitizer is anaminolevulinic acid or methyl aminolevulinate, it is preferentiallyabsorbed by the lesion specific antibody-coated nanoparticles so as toprovide a lesion specific response in the lesion because the area of theskin or mucosa that does not contain photosensitizer will not respond tothe light and will not be affected.

In this further embodiment, riboflavin may be applied topically in abiocompatible solution, the treatment laser may be configured to emitultraviolet (UV) laser light therefrom. For example, topicalapplications of 1% to 2% riboflavin in a biocompatible solution may beapplied as drops for 1 to 20 minutes depending on the concentration ofthe riboflavin and the power of ultraviolet laser light (e.g., 2 mW to20 mW) to activate the riboflavin (the higher the concentration ofriboflavin or the laser power, the less time is needed to concluderadiation). In this further embodiment, the antibody nanoparticles maybe conjugated with riboflavin and polymers, such as polyethylene glycol,chitosan, etc. for systemic, intravenous, or local application.

Also, in this further embodiment, the photosensitizer (e.g., riboflavin)may be used topically in physiological solution to the infected skin ormucosal wound by bacteria, fungi, or other organisms, such as amoebas,that are therapy resistant to medication, alone or with antibiotics orantifungals or the medication, while the infected wound is exposed to UVradiation of 380 mm in wavelength or higher for the appropriate timedepending on the riboflavin concentration and/or the laser energy tocross-link the collagen of the tissue and simultaneously kill theoffending organism. Riboflavin is vitamin B2, and is important formaintenance of the health of the human body. In this further embodiment,the laser wavelength may be chosen depending on the absorption of thephotosensitizer (e.g., the absorption of the riboflavin).

In addition, in this further embodiment, after inducing thephysiological changes and verifying the patient's identification for thePDT procedure using the 3D display, the presence of any lesion on theskin or mucosa, and the size, elevation and the physiological parametersof the lesion are compared before, during, and/or after the procedurewith the results of the previous initial examination. In thisembodiment, the surface of the skin or mucosa is treated with a solutioncontaining pluralities of coated, thermosensitive nanoparticlesconjugated with a polyethylene glycol (PEG) and a thermosensitivepolymer, such as chitosan, a cell penetrating peptide (CPP), aphotosensitizer, and at least one more medication that is released fromthe chitosan, when the temperature of the nanoparticles reaches 41 to 43deg. C. In this embodiment, the temperature of the lesion may bemaintained at 41 to 43 deg. C. to release the medication from thethermosensitive polymer and to prevent a pain sensation. The temperaturemay be raised to 45 to 47 degrees C. for a very short time, whileapplying prophylactically, a topical anesthetic medication in order tokill the cells without creating tissue burn. In another embodiment, thephotosensitizer may be released from the thermosensitive polymer coatingof the nanoparticles when the temperature reaches 38 to 43 deg. C. Also,in another embodiment, the temperature may be raised up to 43 to 45degrees C., 45 to 47 degrees C., or up to 50 deg. C. to kill thecancerous cells of a lesion on the skin mucosa and other accessibleareas. The treatment laser of the laser-imaging system 50, 50′ not onlyactivates the thermosensitive photosensitizers, but also heats up thenanoparticles, etc. Any follow-up treatments are done with the sameremotely-controlled laser and camera system which is equipped with 3Ddisplay described above for accurate recognition and examination of thelesion.

In this further embodiment, when the patient has a skin or mucosallesion or cancer, the combination of the photoacoustic components of thesystem (i.e., the laser and ultrasonic transducer described above)creates not only a 2D image of the surface structure, but using thesystem as photoacoustic tomography provides a 3D image of the lesionproviding gross or microscopic information about the depth that thelesion has penetrated inside the skin or mucosa, while also providingvolumetric information to be measured in the follow-up examination ofthe patient. The 3D image of the lesion may also provide informationabout the effect of the treatment on the lesion in the subsequentfollow-up period, while the data is simultaneously transmitted to adoctor or observer located at the remote site. Also, the photoacousticsystem may be used to provide information about a small region of alarger structure, such as the breast, exhibiting early signs ofpotential cancer in the tissue in an early stage of the disease when thelesion is very small (e.g., 1 to 2 mm dimension) that is difficult toimage by using electromagnetic radiation (i.e., x-ray) or computerizedtomography (CT) scan, etc.

Also, in this further embodiment, the photosensitizer and/ornanoparticles are treated with a laser, for a period of 1 to 30 minutesor more, under observation by processor control, while the laserdelivery system is under the control of the photoacoustic system tocontrol the amount of energy delivered to the lesion for a predeterminedtime. A thermoacoustic response is recorded simultaneously and a thermalmap is created continuously, to prevent heating the tissue beyond thepredetermined temperature of between 39 and 44-47 deg. C. as needed,etc. without creating a thermal burn.

In this further embodiment, the treatment system provides a combinationeffect of PDT if the temperature is maintained below 42 degrees C., andsimultaneously a thermal effect if the temperature is gradually raisedusing a photoacoustic imaging system to keep the temperature below thetemperature in which protein denatures, which is about 50 degrees C.depending on the duration of the application and power of the treatmentlaser.

In this further embodiment, the treatment laser of theremotely-controlled laser system is used for the PDT of a Basal cellcarcinoma, squamatous cell cancer of the skin, angioma, senilekeratosis, precancerous melanosis, melanoma, cancer of the mouth mucosa,throat, nose, vaginal mucosa, cervix, or the uterus. The PDT may also beused to treat photoaging, acne vulgaris, dry eye to enhance meibomiangland secretion, Bowen's disease of the eye or lids, and the PDT mayalso be used to deliver strong light therapy to damage and killbacteria, such as Propionibacterium acnes, bacteria in blepharitis, orto treat infected corneas that are infected by bacteria, fungi, viruses,and amoebas, etc.

System Software Platform

The software system that is employed to control the laser procedure(s)with the laser-imaging systems of the invention includes client/serverarchitecture and a TCP/IP communication protocol. The client/serverarchitecture comprises a computer science paradigm, where clients andservers are deemed separate software processes that can run on the sameor different computers.

In some embodiments, the software recognizes the computer at thesurgeon's site as a client, while the local control module at the remotepatient's site is recognized as a server.

According to the invention, communication by and between the remotecontrol module 64 and the local control module 62 is facilitated via webservices implemented in .NET remoting technology.

When a physician at a remote site sends a control or data acquisitioncommand, the command is first transmitted to the local control module 62(or server) through the .NET remoting interface. In response to thecommand, the local control module 62 controls hardware compartmentsthrough a hardware specific interface (e.g., RS-232C interface, parallelcommunication protocol).

The communication speed between the client and the server will depend onseveral factors such as 1) the distances between the client and theserver, 2) network traffic conditions, and 3) the size of data (e.g.,images) being transmitted.

As will readily be appreciated by one having ordinary skill in the art,the present invention provides numerous advantages compared to prior artmethods and systems for laser coagulation procedures. Among theadvantages are the following:

-   -   The provision of laser-imaging systems, which will significantly        reduce laser transmission and, hence, procedure time. For        example, the length of time for the laser photo-coagulation        treatment for diabetic retinopathy will be reduced from 30-60        minutes per procedure to only two minutes. In general, the        duration of a single laser pulse, plus the time it takes to        perform the subsequent laser application multiplied by the total        number of pulses required is equal to the overall procedure        time. The manner in which the pulses are applied in a        conventional contact system (e.g., using a contact lens        positioned on the cornea in order to see the fundus) requires        the physician (i.e., ophthalmologist) to perform the procedure        laser shot by laser shot (i.e., to place a single laser spot at        a desired location and then move on to the next location). When        performing the procedure on a patient that requires 1000-2000        laser spots, the laser coagulation procedure can easily take 30        minutes or more. The laser coagulation system disclosed herein        uses millisecond laser pulses (and in some cases, less than        millisecond pulses), and each subsequent laser shot is performed        automatically, in some embodiments, by utilizing an oscillating        mirror (e.g., oscillating mirror 220 described above) placed in        the path of the laser beam such that the laser beam is displaced        without requiring any manual means. In addition, the wide angle        camera employed in one or more embodiments described herein        enables the entire retina to be viewed, rather than just a small        portion thereof, thereby further reducing procedure time. As a        result of these enhanced features, the laser coagulation system        described herein substantially reduces the overall procedure        time. In one embodiment, the remote operation module of the        laser coagulation system is configured to perform a fully        automated and continuous laser coagulation procedure over the        entire area of the retina in a period of time no greater than        approximately 2 minutes (or no greater than 2 minutes) in an        actual control mode.    -   The provision of laser-imaging systems, which will also reduce        the probability of error associated with manual surgery (tremors        and misjudgments) via a more precise computerized control        mechanism, with additional fail-safe features, and a wider angle        imaging camera for retina diseases. This offers more choices for        various lasers with different wavelengths, intensities, and        action than was previously possible.    -   The provision of laser-imaging systems, which will also allow a        physician or surgeon to perform a procedure at a remote site,        via a high-speed reliable Internet® connection, thus eliminating        the need for the patient to travel a long distance to be treated        at a specialist's office or, in the case of military field care        or space exploration units, allowing patients to be treated        immediately on-site.    -   The laser coagulation system described herein is in the form of        a non-contact system that does not require the use of a contact        lens or any other device in contact with the eye of the patient.        The laser coagulation system embodied herein also does not        require the physician to indent any portion of the patient's eye        (i.e., no indenter or scleral depressor is required to indent        the peripheral portion of the patient's eye). As a result, the        patient is far more comfortable during the laser coagulation        procedure.

The cameras employed in the systems can also be equipped withappropriate diode lasers and filters for auto-fluorescence photographyand angiography of the retina.

One can also develop a miniature version of the system by using printingtechnology, such as inkjet, to generate and integrate the micro-opticalparts not only on hard substrates, but also on those that are flexible.Having a miniature or micro-system will allow further use of suchtechnology in hard to reach places.

Any of the features or attributes of the above described embodiments andvariations can be used in combination with any of the other features andattributes of the above described embodiments and variations as desired.

Without departing from the spirit and scope of this invention, one ofordinary skill can make various changes and modifications to theinvention to adapt it to various usages and conditions. As such, thesechanges and modifications are properly, equitably, and intended to be,within the full range of equivalence of the following claims.

What is claimed is:
 1. A telemedicine system with dynamic imaging,comprising: a local control system disposed at a first location and acentral control system disposed at a remote site, said remote site beingat a second location, said local control system being operativelycoupled to said central control system by means of a computer network;said local control system including a first computing device with afirst processor, a local control module, and a dynamic imaging system;said central control system including a second computing device with asecond processor and a remote control module; said dynamic imagingsystem including an imaging device operatively coupled to said firstcomputing device, said imaging device configured to capture images of abody portion of a patient over a predetermined duration of time so thata displacement of said body portion of said patient is capable of beingtracked during said predetermined duration of time, said captured imagesincluding an initial captured image containing first image data that isproduced without requesting said patient to perform any task, and saidcaptured images further including a subsequent captured image containingsecond image data that is produced as a result of said patient beinginstructed to perform a task that induces one or more detectablephysical and physiological changes in said body portion of said patient,said first computing device being programmed to determine saiddisplacement of said body portion of said patient over saidpredetermined duration of time using said captured images, and tocompare said displacement of said body portion of said patient over saidpredetermined duration of time to a reference displacement of said bodyportion of said patient acquired prior to said displacement so that aplurality of induced physical and physiological dynamic changes in saidbody portion of said patient are capable of being assessed for thepurpose of identifying said patient or evaluating physiological changesin said body portion, and said first computing device being furtherprogrammed to compare using a subtraction algorithm said plurality ofinduced physical and physiological dynamic changes to said body portionof said patient captured by said imaging device at a first time to saidplurality of induced physical and physiological dynamic changes to saidbody portion of said patient captured by said imaging device at a secondsubsequent time, and to determine if said plurality of induced physicaland physiological dynamic changes captured by said imaging device atsaid first time substantially matches said plurality of induced physicaland physiological dynamic changes to said body portion of said patientcaptured by said imaging device at said second subsequent time so as toverify the identity of said patient; wherein the body portion is a faceand the subtraction algorithm eliminates unchanged parts of the face,while maintaining changed parts of the face to enhance the processingspeed.
 2. The telemedicine system according to claim 1, wherein saidimaging device of said dynamic imaging system is in the form of a lightfield camera.
 3. The telemedicine system according to claim 1, whereinsaid dynamic imaging system is in the form of a dynamic facialrecognition system, and wherein the face or a portion of the face ofsaid patient is imaged in a two-dimensional or three-dimensional manner.4. The telemedicine system according to claim 3, wherein said firstcomputing device is further programmed to reconstruct a complete facefrom the portion of the face captured by the dynamic facial recognitionsystem using artificial intelligence and/or a machine learningalgorithm.
 5. The telemedicine system according to claim 3, wherein saiddynamic imaging system further comprises a fingerprint sensor configuredto capture multispectral images of a finger of said patient, said firstcomputing device being specially programmed to determine surfacefeatures and subsurface features of said finger of said patient usingsaid multispectral images.
 6. The telemedicine system according to claim3, wherein said dynamic imaging system further comprises a handwritingrecognition sensor configured to capture handwriting of said patient,said first computing device being specially programmed to determine anidentity of said patient based upon said handwriting of said patientcaptured by said handwriting recognition sensor.
 7. The telemedicinesystem according to claim 3, wherein said first computing device isfurther programmed to utilize a fuzzy logic mathematical algorithm inorder to differentiate an identity of two similar images captured bysaid dynamic facial recognition system.
 8. The telemedicine systemaccording to claim 3, wherein said first computing device is furtherprogrammed to add or subtract features from the face of said patientcaptured by said dynamic facial recognition system in order tofacilitate the identity recognition of said patient when an appearanceof said patient changes over time, said added or subtracted featuresselected from the group consisting of addition of hair, removal of hair,addition of facial hair, removal of facial hair, addition of jewelry,removal of jewelry, addition of eyewear, and removal of eyewear.
 9. Thetelemedicine system according to claim 3, wherein said first computingdevice is further programmed to change features of the face of saidpatient captured by said dynamic facial recognition system in order tofacilitate the identity recognition of said patient when an appearanceof said patient changes over time, said changed features selected fromthe group consisting of skin color, hair color, eye color, presence orabsence of hair, presence or absence of facial hair, presence or absenceof jewelry, and presence or absence of eyewear.
 10. The telemedicinesystem according to claim 3, wherein said first computing device isprogrammed to compare one or more first reference digital images of saidpatient captured by said imaging device of said dynamic facialrecognition system at a first time to one or more second digital imagesof said patient captured by said imaging device of said dynamic facialrecognition system at a second subsequent time using stable features ofthe face of said patient, and to determine if said one or more seconddigital images of said patient substantially matches said one or morefirst reference digital images of said patient, said stable featuresselected from the group consisting of a nose position, an eye position,a mouth position, spacing between eyes, spacing between the nose and oneor more of the eyes, spacing between the nose and the mouth, eyefeatures, nose features, mouth features, ear features, and geometricfacial features.
 11. The telemedicine system according to claim 3,wherein said first computing device is further programmed to changeemotionally affected features on an image of the face of said patientcaptured by said dynamic facial recognition system in order tofacilitate the identity recognition of said patient when an emotionalstate of said patient is different.
 12. The telemedicine systemaccording to claim 1, wherein said changed parts of said face of saidpatient for which said displacement is determined comprises a mouth ofsaid patient; and wherein said telemedicine system further comprises avoice recognition sensor configured to capture speech waveformsgenerated by said patient so that said speech waveforms are capable ofbeing correlated with a mouth configuration of said patient generatedfrom said captured images acquired using said imaging device, therebyenabling both audial and visual attributes of said patient to be takeninto account for identification purposes.
 13. The telemedicine systemaccording to claim 1, wherein said imaging device of said dynamicimaging system is in the form of a multispectral camera or hyperspectralcamera configured to capture multispectral or hyperspectral images ofsaid body portion of said patient so that surface features andsubsurface features of said body portion of said patient are capable ofbeing analyzed.
 14. The telemedicine system according to claim 13,wherein said first computing device is operatively coupled to a visualdisplay device and one or more speakers, said first computing devicebeing programmed to display a range of images and/or videos on saidvisual display device and/or play a variety of different sounds on saidone or more speakers to evoke different emotional states of saidpatient, and said first computing device being further programmed torecord dynamic changes in said body portion of said patient with saidmultispectral camera or hyperspectral camera while evoking the differentemotional states of said patient so that a response of said patient tosaid different emotional states is able to be analyzed.
 15. Atelemedicine system with dynamic imaging, comprising: a local controlsystem disposed at a first location and a central control systemdisposed at a remote site, said remote site being at a second location,said local control system being operatively coupled to said centralcontrol system by means of a computer network; said local control systemincluding a first computing device with a first processor, a localcontrol module, and a facial recognition system; said central controlsystem including a second computing device with a second processor and aremote control module; said facial recognition system including animaging device operatively coupled to said first computing device, saidimaging device configured to capture one or more images of a face of apatient, said captured images including an initial captured imagecontaining first image data that is produced without requesting saidpatient to perform any task, and said captured images further includinga subsequent captured image containing second image data that isproduced as a result of said patient being instructed to perform a taskthat induces one or more detectable physical and physiological changesin said face of said patient, said first computing device beingprogrammed to compare using a subtraction algorithm said plurality ofinduced physical and physiological dynamic changes to said face of saidpatient captured by said imaging device of said facial recognitionsystem at a first time to said plurality of induced physical andphysiological dynamic changes to said face of said patient captured bysaid imaging device of said facial recognition system at a secondsubsequent time, and to determine if said plurality of induced physicaland physiological dynamic changes captured by said imaging device atsaid first time substantially matches said plurality of induced physicaland physiological dynamic changes to said face of said patient capturedby said imaging device at said second subsequent time so as to verifythe identity of said patient; wherein the subtraction algorithmeliminates unchanged parts of the face, while maintaining changed partsof the face to enhance the processing speed.
 16. The telemedicine systemaccording to claim 15, wherein said first computing device is furtherprogrammed to receive consent from said patient for telemedicinecommunication by receipt of verbal confirmation, a confirmatory headmotion, a confirmatory hand motion, a confirmatory eye motion, and/or byreceipt of a fingerprint from said patient.